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ABSTRACT 


Human error has been identified as a factor in virtually every major maritime 
mishap over the past decade. The Department of Defense (DoD) currently 
employs the Human Factors Analysis and Classification System (HFACS) 
taxonomy to identify and quantify human error in major mishaps. HFACS divides 
errors into categories, sub-codes, and nano-codes. The generic nature of DoD 
HFACS raises the question of whether or not a domain-specific version for the 
surface Navy could be applied more consistently. Twenty-eight subjects (14 
Surface Warfare Officers (SWOs) and 14 non-SWOs) employed either DoD 
HFACS or a developmental maritime domain specific version, HFACS-M, to 
classify findings in a National Transportation Safety Board (NTSB) maritime 
accident investigation. Fleiss’ Kappa was used to determine inter-rater reliability 
among subjects. The results of this study revealed that SWOs using HFACS-M 
had a higher inter-rater reliability (10.9%, 7.3%, and 6.5%) at every classification 
level than non-SWOs. HFACS-M itself was also shown to have a slightly higher 
overall inter-rater reliability (5.7%, 7.4%, and 3.6%) than DoD HFACS. The 
research concluded that although HFACS-M performed well, further testing is 
necessary to validate it. 
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EXECUTIVE SUMMARY 


An analysis of accident investigations throughout the surface Navy suggests that 
nearly every mishap contains some level of human error. To identify mishaps 
properly for mitigation and elimination, the Navy must have an effective error 
classification system. The Department of Defense (DoD) has implemented the 
Human Factors Analysis and Classification System (HFACS) to address this very 
issue. HFACS asserts that errors arise in four distinct categories: organizational 
influences, supervision, existing preconditions, or the very acts themselves. Each 
category is divided into sub-codes, and each sub-code into nano-codes to 
identify specific errors. HFACS was originally developed for naval aviation but 
has been adapted for use in all branches of service. Several published studies 
suggest that domain-specific error classification systems may lead to higher 
inter-rater reliability. To this end, a maritime specific version of HFACS, HFACS- 
M, was developed. 

Twenty-eight students from the Naval Postgraduate School (14 Surface 
Warfare Officers (SWOs) and 14 non-SWOs) received training on either DoD 
HFACS or HFACS-M and then were asked to employ them in a real-world 
scenario. Subjects were asked to classify 11 findings in a National Transportation 
Safety Board maritime accident investigation using one of the taxonomies to 
assign an appropriate nano-code. The subjects’ responses were compiled into 
two tables, one for HFACS, and one for HFACS-M. The tables were then 
separated between SWOs and non-SWOs. Inter-rater reliability was calculated 
for each error classification taxonomy using Fleiss’ Kappa. Overall inter-rater 
reliability and inter-rater reliability for SWOs and non-SWOs were calculated. 
This process was repeated at the sub-code and category level. 

Analysis showed that, of the two taxonomies, HFACS-M had a slightly 
higher overall inter-rater reliability at every level (5.7%, 7.4%, and 2.8%) than 
DoD HFACS. When using the domain-specific taxonomy, SWOs displayed a 
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higher inter-rater reliability (10.9%, 7.4%, and 6.5%) than non-SWOs. Non-SWOs 
did, however, have a slightly higher inter-rater reliability (10.2%, 4.3%, and 8.4%) 
when employing DoD HFACS. 

The research concluded that, in this particular study, SWOs performed 
slightly better at every level of analysis than non-SWOs when applying the 
domain-specific error classification taxonomy. It was also found that HFACS-M 
had a slightly higher overall inter-rater reliability at each level than DoD FIFACS. 
Due to a small sample size and lack of trained raters, it cannot be stated 
conclusively that HFACS-M is a significantly better method for classifying error in 
the surface Navy. It can be concluded, however, that the results of this study 
support the need for further research. Additionally, the Navy should attempt to 
address the gaps in latent distal errors and maintenance-specific errors. 
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I. INTRODUCTION 


A. OVERVIEW 

Human error has been a cause in virtually every significant mishap within 
the surface Navy for the past several decades. Based on Naval Safety Center 
data from January 1992 through December 1996, human error was found to be a 
factor in 100% of all recorded incidents (Lacy, 1998). As such, the reduction of 
human error has been a key focus of the Navy, as well as other organizations for 
many years. 

Reason’s research into human error brought him to the belief that in a 
perfect world, mishaps are nearly always preventable. He saw each accident as 
an event that could be prevented at different points. Much like slices of Swiss 
cheese, these layers were filled with holes (Figure 1) in the real world. Reason 
asserted that these holes were due to some combination of latent and active 
failures (Reason, 1997). 



Some ‘holes’ 
due to active 
\ failures 


/ Defences 
/ in depth 


Other 'holes’ 
due to latent 
conditions y 


Figure 1. Reason’s original “Swiss Cheese” model 

(From Reason, 1997) 


Reason’s theory was a catalyst for the team of Shappell and Wiegmann, 
who took the basics of the theory and developed a method for attributing 
causality in accidents (Shappell & Wiegmann, 2001). The Department of Defense 
(DoD) Human Factors Analysis and Classification System (HFACS) is a 
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taxonomy for classifying mishaps. Using the “Swiss cheese” model as a starting 
point, Shappell and Wiegmann assigned names to each of the layers, or levels 
(Figure 2). DoD HFACS consists of four levels: organizational influences, 
supervision, preconditions, and acts; the holes within each of which lead to the 
eventual mishap. At each level, the taxonomy is broken down into categories, or 
sub-codes, and then into nano-codes (Shappell & Wiegmann, 2001). The surface 
Navy currently uses DoD FIFACS in classifying all its major mishaps (Department 
of Defense, 2005). 



Mishap 


Figure 2. The “Swiss Cheese” model—HFACS version 

(After Reason, 1990; DoD, 2005) 


Since its creation, HFACS has been widely researched, with more than 
90 articles published on the subject. The research surrounding HFACS is 
effectively split into two categories, DoD HFACS and hybrid versions of DoD 
HFACS. Next, the research is further broken down into analysis using the 
HFACS sub-codes and analysis using nano-codes. Of these four possible 
combinations, the most prevalent research concerns DoD HFACS at the sub¬ 
code level, while the least common examines non-DoD HFACS at the nano-code 
level. 
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The majority of HFACS research presupposes the mishap ratings are 
accurate. Many studies use a consensus method whereby a group of experts 
discusses the factors of the mishap before arriving at a final decision, much like 
what would occur at a mishap investigation board. Coding at the categorical level 
has been shown to have less inter-rater error, presumably due to the small 
number of sub-codes (19) compared to the large number of nano-codes (144). 

Not all researchers presuppose sufficient inter-rater reliability, however. 
O’Connor has published several papers testing the reliability, utility, and validity 
of HFACS using trained raters, simulated mishap boards, and experienced 
aviators. O’Connor’s findings suggest the need for more robust HFACS training, 
particularly for end users, and a more robust verification and validation process 
for the evaluation system being used—HFACS or otherwise (O’Connor, 2008; 
O’Connor, Walliser, & Philips, 2010; O’Connor & Walker, 2011). 

Salmon, Cornelissen, and Trotter (2012) also questioned HFACS’ 
reliability. The researchers conducted a comparison of several accident analysis 
methods, including Accimap, HFACS, and STAMP. Although they concluded that 
HFACS was a better system to use in a large organization, such as the DoD, 
they raised questions about HFACS’ reliability and were concerned about the 
lack of domain specificity outside of aviation. 

Finally, in one of the most recent studies utilizing HFACS, Griggs (2012) 
investigated mishaps within the commercial maritime sector and applied HFACS 
to a series of 48 mishaps. His research determined that, “in order to improve the 
reliability of HFACS, the taxonomy needs to be relevant to the maritime 
community” (Griggs, 2012, p. 85). 

B. BACKGROUND 

Accidents are an unfortunate reality within the United States (U.S.) Navy, 
and repair funds are allotted each year to cover the costs. Unfortunately, as 
technology advances, the cost to repair systems involved in these mishaps 
increases exponentially. 
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Failure to learn from past mishaps all but ensures that those mishaps will 
be repeated in time. To identify and prevent the root cause of hazards that result 
in major mishaps properly, the Navy convenes safety investigation boards (SIB) 
for each of the following: 

1. All on-duty Class A mishaps on or off a government 
installation (while performing official duties); in 
commissioned and pre-commissioned U.S. Navy ships after 
delivery; United States Naval Ships (USNS) with federal 
civilian mariner crews in the Military Sealift Command 
(MSC); Navy-owned experimental and small craft; and the 
ship's embarked equipment, boats, and landing craft, or 
leased boats. 

2. Military death that occurs during or as the result of a medical 
event that occurs within one hour after completion of any 
command-directed remedial physical training (PT), physical 
readiness test (PRT), physical fitness testing (PFT), physical 
fitness assessment (PFA) or command-sponsored activity 
during normal working hours regardless of any pre-existing 
medical condition. 

3. On-duty injury where death or permanent total disability is 
likely to occur, or where damage estimates may be expected 
to exceed one million dollars. 

4. Hospitalization, beyond observation, of three or more 
personnel, at least one of who is a DoD civilian, involved in a 
single mishap. 

5. All explosives mishaps, all ordnance impacting off range and 
all live fire mishaps resulting in an injury. 

6. Any mishap that a controlling command (as defined in 
paragraph 1005.6) determines requires a more thorough 
investigation and report, beyond that provided by a 
command’s safety investigator. (Department of the Navy, 
p. 6-1,2005) 

Upon concluding, each SIB produces a list of findings and follow-on 
recommendations. The SIB analyzes these findings to determine which hazards 
were causal to the mishap, and which were contributory (did not directly cause 
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the incident). The SIB then converts the causal and contributory factors to nano¬ 
codes using HFACS (Department of the Navy, p. A-15, 2005). 

The instruction that governs the SIB process provides guidance with 
respect to the board’s composition. The composition is required to be as follows: 

1. Minimum composition of an SIB is three members; however, 
five is preferred. 

2. The appointing authority and senior member of the board 
can confer and agree on board appointees based on the 
type and severity of the mishap. 

3. For afloat mishaps, all members must be commissioned 
Officers. If the mishap involves more than one naval 
command, a Navy, Marine, or MSC representative as 
appropriate, shall be a member of the SIB. 

4. The senior member appointed to the SIB shall not be from 
mishap command. All SIBs shall consist of: 

a. A senior member, who shall be a commissioned 
Officer (0-5 or above), a senior civilian (GS-13 or 
higher), or a senior official in MSC as appropriate. 

(1) A military senior member of a Navy SIB shall 
be senior to the commanding officer of the 
command or unit involved in the mishap. 

(2) The senior member of a Marine Corps SIB 
shall be a Marine Corps officer or a senior 
civilian (GS-13 or higher), and shall be equal to 
or senior in grade to the commander of the 
mishap unit. 

(3) In cases where the senior member requirement 
cannot be met, the appointing authority shall 
request a waiver from the appropriate 
controlling command. 

b. At least two additional members (one of whom could 
be a subject matter expert (SME) on equipment, 
systems or procedures). (DON, p. 6-3, 2005). 
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These requirements present several potential issues. First, none of the members 
is required to have any background or training in HFACS or investigative 
procedures (Department of the Navy, p. 6-3, 2005). This board composition 
policy creates the potential for incorrect HFACS coding. Secondly, HFACS, now 
called DoD HFACS, is used throughout all branches of military service and 
contains generic and non-domain specific codes, which leads to the greater 
likelihood of erroneous coding. 

C. PROBLEM STATEMENT 

The HFACS taxonomy converts qualitative mishap data to categorical 
data for the purpose of analysis. The results of these analyses are used to help 
decision makers determine how money should be spent to prevent future 
mishaps. If a mishap is coded incorrectly, that information is entered into a 
database and could lead to incorrect assumptions when analyzed. Given the low 
inter-rater reliability found in several studies using DoD HFACS (as low as 36% 
overall and as low as 22.5% for causal factor agreement), it is imperative that the 
reasons for this disparity be investigated, and methods to improve reliability be 
explored (Baysari, Caponecchia, McIntosh, & Wilson, 2009; O’Connor, 2008; 
O’Connor et al., 2010; O’Connor et al., 2011). 

This study seeks to gain insight into the existing claims of sub-optimal 
inter-rater reliability when using HFACS (Baysari et al., 2009; O’Connor, 2008; 
O’Connor et al., 2010; O’Connor et al., 2011). To inform decision makers 
correctly about where to spend tax dollars, mishap coding must be accurate. This 
study also introduces a maritime-specific version of HFACS for use in the surface 
Navy, referred to as HFACS-M (maritime), in an effort to observe whether or not 
a domain-specific version of HFACS results in increased inter-rater reliability. 
The study also considers the role of training in HFACS coding. 

D. OBJECTIVES 

The purpose of this thesis is to compare the inter-rater reliability, usability, 

and validity of HFACS and HFACS-M, which is a modification to HFACS 
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developed by the author and tailored specifically to surface ship mishaps. The 
objective is to use the results to identify any possible gaps in the human error 
taxonomies for the surface Navy. The results will lead to updated taxonomies to 
ensure that the U.S. Navy is able to identify human error correctly and reduce the 
number of mishaps in the future. 

E. RESEARCH QUESTIONS 

To identify potential gaps, overlaps, and errors within HFACS and 
HFACS-M, this study attempts to answer the following research questions. 

• Do Surface Warfare Officers (SWOs) and Non-SWOs show the 
same consistency when applying DoD HFACS? 

• What errors, overlaps, or gaps, if any, currently exist in DoD 
HFACS? 

• Does a tailored version of HFACS result in increased inter-rater 
reliability when classifying mishaps within the surface Navy? Why 
or why not? 

F. SCOPE AND LIMITATIONS 

This research was limited to the results of the statistical analysis of the 
data collected from two case studies. Although this research focused on accident 
analysis within the surface Navy, Naval Safety Center data for major afloat 
mishaps was restricted. This research focused on the HFACS classification of 
Class A Mishaps as defined by the current version of OPNAVINST 5102, the 
Navy and Marine Corps Mishap and Safety Investigation, Reporting, and Record 
Keeping Manual (Department of the Navy, 2005). 

G. HSI 

This section discusses the applicable domains of HSI which pertain to this 
research. More specifically, the areas of Manpower, Personnel and Training 
(MPT) and Human Factors Engineering (HFE) are considered in this thesis. 
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1. Manpower, Personnel, and Training 

The manpower domain of HSI seeks to develop systems that “optimize 
manpower and keep human resource costs at affordable levels” (DAU, 2009). An 
example of a manpower issue is determining the optimal number of sonar 
technicians required onboard a Guided Missile Destroyer (DDG) to fill three 
watch sections. Manpower is an important factor in mishap investigation. Many 
times human error occurs because Sailors are overworked or severely stressed. 
Overwork in military settings can often be attributed to the improper manning of a 
system. Improper manning has been shown to lead directly to an increase in 
safety related mishaps (Lazzaretti, 2008). 

The personnel domain of HSI differs from manpower in that it focuses on 
“human aptitudes (i.e., cognitive, physical, and sensory capabilities), knowledge, 
skills, abilities, and experience levels that are needed to properly perform job 
tasks” (Defense Acquisition University, 2009). From a human error perspective, 
the selection of Sailors and Officers with inappropriate qualifications and 
experience levels is tantamount to ensuring a mishap will occur in due time. 

The DAU defines training as “any activity that results in enabling users, 
operators, maintainers, leaders and support personnel, to acquire, gain or 
enhance knowledge, skills, and concurrently develops their cognitive, physical, 
sensory, team dynamics and adaptive abilities to conduct joint operations and 
achieve maximized and fiscally sustainable system life cycles” (Defense 
Acquisition University, 2009). As systems employ more technology, the number 
of personnel needed to operate, maintain, and support the system should 
decrease. To balance this, however, more training is required. In the surface 
Navy, command wide, departmental, and divisional training provide invaluable 
knowledge to shipboard personnel. Failure to provide specific training leads to 
human error, which leads to mishaps. 
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2. Human Factors Engineering 

HFE is the HSI domain that supports many of the other domains. HFE 
seeks to ensure systems “capitalize on and do not exceed the abilities (cognitive, 
physical, sensory, and team dynamic) of the user population” (Defense 
Acquisition University, 2009). In systems that have had FIFE applied properly 
during the design process, a significant reduction often occurs of either cognitive 
or physical workload, or both. Consequently, failing to apply proper FIFE during 
system development can be the cause of mishaps due to physical or cognitive 
overload of the human. 

H. ORGANIZATION 

This thesis is divided into six chapters. Chapter I provides a synopsis of 
human error research and some background on the development and uses of 
FIFACS. Chapter II provides a review of the available research on FIFACS. 
Chapter III explains how the FIFACS-M taxonomy was developed and the 
methodology used to evaluate FIFACS and FIFACS-M. Chapter IV provides an 
analysis of the resulting data, and addresses the significant issues uncovered by 
the research. Chapter V discusses the implications of the study’s results. Chapter 
VI offers conclusions and recommendations for future research. 
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II. LITERATURE REVIEW 


A. MISHAPS 

Mishaps comprise the largest unintended costs for the surface Navy 
today. In times of financial constraints, the Navy cannot afford to spend tax 
dollars on incidents that should not have occurred, given proper planning, 
training, and preparation. Mishaps, however, are an inevitable part of any 
organization. As Reason noted, organizational accidents are “comparatively rare, 
but often catastrophic, events that occur within complex modern technologies” 
(Reason, 1997, p. 1). 

Reason explained his theory of how mishaps occur using the terms 
hazards, defenses, and losses (Figure 3). 



Figure 3. Relationship between hazards, defenses, and losses 

(From Reason, 1997) 


A hazard is a potential mishap or something that could go wrong if not 
prevented. In the surface Navy, such a hazard is a ship running aground, or 
colliding with another ship. Losses are the result of an undefended or 
unprevented hazard. Losses come in the form of injury or loss of life to personnel 
or damage to equipment. 

Defenses, on their most basic level, are created to prevent losses and, as 
Reason explains, serve one or more specific functions. First, they “create an 
understanding and awareness of local hazards” (Reason, 1997, p. 7). In the 
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surface Navy, these defenses could be a Local Notice to Mariners report, Coast 
Guard broadcast or warning, or even a Naval Criminal Investigative Service 
(NCIS) port brief before sailors go ashore. Second, they provide guidance on 
safe operation that could be a Navy standard operating procedure (SOP), 
maintenance requirement card (MRC), or a safety checklist. Third, Reason 
asserts that defenses also “provide alarms and warnings when danger is 
imminent” (Reason, 1997, p. 7) that include tank low-level alarms, smoke 
detection and heat detection sensors, and chemical detection units on 
engineering equipment. Fourth, these defenses will return the system to a normal 
operating state following an emergency, which includes releasing fire zone doors 
following a fire, or recycling vent dampers following a missile launch. Defenses 
also act as barriers, primarily physical ones, to prevent the loss from actually 
occurring or to mitigate it. In the fleet, these types of defenses could be a firing 
cutout, which prevents the system from engaging the superstructure of a ship, or 
any redundant safety measure to prevent spills of chemicals or fuels. In some 
situations, defenses are needed “to contain and eliminate the hazards should 
they escape this barrier” (Reason, 1997, p. 7), which can be a floating oil barrier 
placed around a ship when it pulls alongside a pier, or an agent, such as Halon 
or aqueous film forming foam (AFFF), which are designed to eliminate or contain 
fires. Finally, defenses provide a way to exit an area or save human lives in the 
event the primary and secondary barriers fail (Reason, 1997) that can be 
implemented through escape trunks or scuttles, first aid or eyewash stations, life 
rafts, and distress beacons. 

Defenses, however, are not perfect in practice. Defenses are often 
operated by humans who are prone to error. Additionally, many defenses require 
some amount of warning time to be fully activated or effective. To this end, 
Reason developed the concept of the “Swiss Cheese Model” (Reason, 1997). 

Figure 1 shows the basic concept of the Swiss cheese model, which 
follows from his initial established relationship between, hazards, defenses, and 
losses. Defenses (Swiss cheese) have holes resulting from active or latent 
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failures in the defenses. As Reason explains, accident causation is dynamic, and 
can be triggered locally, occur from defects in the defenses themselves, or be 
caused by atypical conditions (Reason, 1990). 

Reason developed a model to explain how the hazard to loss process 
worked in relation to latent and active failures (Figure 4). The triangle portion of 
the figure represents the factors or conditions leading up to an event 
(represented by the rectangle at the top). Latent or active, these failures work 
together to create an error chain that eventually resulted in a loss (Reason, 
1997). 



Figure 4. Stages in the development and investigation of an organizational 

accident (From Reason, 1997, p. 17) 

As an example, a ship has been extended on deployment, which has 
taken its toll on the crew (latent, organizational factor). The helmsman has had 
insufficient sleep (latent, local workplace factor) as he steers the ship late at 
night. The Conning Officer is supposed to be watching the navigation situation, 
but is preoccupied by his upcoming Surface Warfare Officer (SWO) board, and is 
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not paying attention (active, local workplace factor). The ship is transiting a 
narrow channel and strays into shoal water on its starboard side due to the 
helmsman nodding off with no one paying attention to him (active, unsafe act). 
The Officer of the Deck (OOD) and Junior Officer of the Deck (JOOD) stand out 
on the port bridge wing and have a discussion about the NCAA Basketball 
Tournament currently going on (active, local workplace factor). The resulting loss 
is a grounding of the ship, millions of dollars in repairs, and the end of several 
careers. 

B. ACCIDENT INVESTIGATION 

In the event a major mishap does occur within the surface Navy, an 
investigation is required. The purpose of this investigation is to “reveal hazards 
that could cause future mishaps” (Department of the Navy, 1 -1,2005). 

In a perfect world, Reason’s model (Figure 5) simply works in reverse, in 
that an investigation is concerned. A loss is realized (mishap itself), and then 
analyzed to determine what latent and active failures in the layers of defenses 
occurred to identify the potential hazard (Reason, 1997). 

The unfortunate reality, as Schmorrow accurately explained, is that 
accident investigation does not happen in a perfect world, or vacuum. Far from it, 
in fact. Accident investigation is influenced by many factors including (but not 
limited to) inherent bias, time constraints and the post-hoc nature of the 
investigation itself, as well as the accident-reporting model being used 
(Schmorrow, 1998). 

According to Schmorrow, “the perceptions of individual accident 
investigators can confound the goals of an accident investigation” (Schmorrow, 
1998, p. 14). For instance, a civilian engineer looking at a collision will almost 
instinctively focus on the most familiar systems. This bias could lead to 
overlooking HSI issues that actually contributed to the mishap. Additionally, 
previous experience or inexperience with particular types of accidents can lead to 
incorrect conclusions. If experience tells the investigator that 80-90% of 
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accidents have been attributed to human error, a predisposition may find fault in 
crew members where it may not exist (Schmorrow, 1998). 

Time and timing also play significant parts in the investigatory process. As 
investigations are generally only conducted in relation to catastrophic events, 
such as the collision or grounding of a ship, the pressure to conduct the 
investigation in a timely manner is significant. The Navy, as would any 
organization, wants to find out what went wrong to prevent that loss from 
happening again, which can cause undue stress on the investigators, and 
potentially lead them to overlook or miss something. Additionally, the post-hoc 
nature of the process itself can hamper the truth. If a member or members of the 
bridge watch team were intoxicated at the time of the incident, but not given a 
breathalyzer test at the time, it may not be possible to prove that alcohol 
contributed to the incident. Additionally, part of the nature of the Navy is the sea, 
the very environment in which it operates. Tides and currents can quickly and 
easily wash away evidence that may be vital to recreate the story of what 
happened. 

The last major factor of an accident investigation is the accident-reporting 
model being used. Various forms, models, formats, and procedures are prevalent 
in the field of accident investigation. This raises at least two key questions. First, 
what if the investigation produces results not consistent with the reporting model? 
Second, if the model tells the investigators what they are “supposed” to find, will 
they then shape their results to fit that model? 

C. HFACS 

HFACS was developed by Shappell and Wiegmann and is based on 
Reason’s (1990) previously described model of human error. The purpose of 
FIFACS is to establish a “comprehensive, user-friendly tool for identifying and 
classifying the human causes of aviation accidents” (Shappell & Wiegmann, 
2001). Originally developed for use in the Naval Service (Navy and Marine Corps 
aviation), HFACS is now required to be used across all branches of service for 
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the classification of human error in accidents (DoD, 2005). The original version 
has been adapted to an all-inclusive version that can be used in land, air, 
surface, and sub-surface accidents. 

Several other methods of accident classification currently in use in the 
civilian sector are worth mentioning. Accimap was developed by Rasmussen in 
1997 and divides safety within a given system into levels consisting of 
government policy and budgeting, regulatory bodies and associations, local area 
government planning and budgeting, technical and operational management, 
physical processes and actor activities, and equipment and surroundings. This 
method of error analysis is generic and does not use a taxonomy (Salmon et al., 
2012 ). 

STAMP is a second method of mishap classification, and focuses on 
control as the primary reason for failures. These controls are divided into 
managerial, organizational, physical, operational, and manufacturing. The final 
description produced by this method highlights the overall control structure of a 
system, and which parts yielded the failure in question (Salmon et al., 2012). 

As HFACS is the only error taxonomy currently in use by the DoD, it will 
be the focus of this research. 

1. Structure and Usage 

HFACS bridges the gap between Reason’s theory and the actual practice 
of classifying human error in accident investigation. To this end, the HFACS 
framework divides Reason’s model into four levels of human error: organizational 
influences, supervision, preconditions, and acts. Listed under each of these 
categories are nano-codes that allow for greater specificity as to the nature of the 
latent or active failure that contributed to the mishap. 
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a. Organizational Influences 

Organizational Influences (Figure 5) fall under Reason’s latent 
failures. Decisions made by numbered Fleet Commanders, Type Commanders 
(TYCOMS), and even Immediate Superiors in Command (ISICs) can eventually 
lead to mishaps. 



Figure 5. Organizational factors influencing accidents 


Resource and Acquisition Management refers to decision making 
regarding equipment purchases, upgrades, upkeep, and general fiscal 
management. Examples of nano-codes include inadequate personnel recruiting 
policies, insufficient support facilities and equipment, failure to provide sufficient 
funding, failure to remove or upgrade antiquated equipment, and purchasing 
poorly designed or unsuitable equipment (Naval Safety Center, 2007). 

Organizational climate refers to the “atmosphere” within an 
organization. A command’s climate often tells a great deal about it. 
Organizational climate issues influencing mishaps may include over-confidence 
in equipment, unclear organizational structure, and undue pressure or demand 
for mission accomplishment (Naval Safety Center, 2007). 

Similarly, the processes of an organization may set up commands 
in the lower echelons for failure. Unsafe conditions due to high operational 
tempo, inadequate procedural guidance, unsatisfactory program management, or 
lack of formal training can all have long-term and unintended impacts (Naval 
Safety Center, 2007). 
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b. Supervision 

Supervision, shown in Figure 6, more specifically at the command 
level, has a direct impact on safety and risk management within that command. 
Supervisors failing to adhere to rules and regulations, as well as failing to require 
their subordinates to do the same, may set their commands up for catastrophic 
failure. 



Figure 6. Categories of unsafe supervision 


Inadequate supervision and leadership can quickly lead to disaster. 
In high stress situations or instances when subordinates are relatively unfamiliar 
with the unfolding situation (inexperienced), supervision is instrumental in 
preventing potential mishaps. Lack of training, guidance, policy, and even 
personality conflicts are examples of inadequate supervision. 

Risk arises when inappropriate operations are planned. If a 
supervisor selects an individual without the requisite experience level for a task, 
authorizes an unnecessary hazard, or directs actions to be taken outside the 
capabilities of equipment, a mishap is likely to follow. 

Similarly, it is incumbent upon supervisors to correct issues brought 
to light. Failing to correct risky behavior or unsafe practices by subordinates can 
have catastrophic consequences. 

Lastly, violating or intentionally disregarding guidance or policies 
creates undue risk within a command. Failing to enforce rules, espousing “tribal 
knowledge” over written instructions, or directing violations of standard policies, 
create risk that can lead to eventual disaster (Naval Safety Center, 2007). 
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c. 


Preconditions 


Latent or potential hazards exist all around. As Figure 7 helps 
illustrate, in a high-tempo and complex organization, such as the surface Navy, 
both the physical and technical environment can play significant roles in causing 
mishaps. Personal issues existing within individuals and among individuals in an 
organization also can contribute to mishaps. 
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Figure 7. Categories of preconditions for unsafe acts 
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(1) Environmental Factors. On the environmental side, 
weather and the ambient environment within a ship are rife with latent hazards. In 
addition, flaws in equipment design can lie dormant for months or years but can 
eventually contribute to a mishap. Cold and heat stress, restricted visibility, 
lighting or backscatter, equipment interface issues (HFE), and instrumentation 
and warning issues are possible technical preconditions for a mishap. 

(2) Condition of Individuals. The largest set of 
preconditions comes, unsurprisingly, from individuals. Such factors include 
physical or mental limitations, cognitive factors, adverse physical states, psycho- 
behavioral factors, and perceptual factors. 

Physically, humans have limitations. Be it with memory, 
learning rate, coordination, or even body size, an individual’s capabilities, or lack 
thereof, can be a precursor for failure, given the right situation. 

Issues with how an individual perceives a given situation can 
prove to be risky as well. Spatial disorientation, coupled with misinterpreting or 
misreading instruments, and misperceiving a changing environment can cause 
individuals to respond incorrectly for a given situation, eventually leading to 
disaster. 

Even an individual’s personality, motivation level, and other 
psychosocial issues or psychological disorders can prove to be a source of risk 
given the proper situation. Emotional state, excess aggression, overconfidence, 
and complacency are potential factors within individuals that can impact 
decisions and create added risk. 

The final precondition within individuals deals directly with 
physiological states. Existing medical or physiological conditions include the 
effects of prescribed drugs, overexertion, motion sickness, and dehydration. 


21 



(3) Personnel Factors. The last category of preconditions 
exists among personnel. Communication, coordinating and planning, as well as 
self-imposed stress, must be considered factors during accident investigation, as 
they can play a major role in mishaps. Self-imposed stress takes many forms. 
Alcohol, improper diet, illegal drugs, and even the fitness level of personnel can 
all become precursors for serious incidents, given the right prevailing 
circumstances. 

One of the largest and most common sections of precursors 
is those of coordination, communication and planning. Lack of assertiveness, 
failure to communicate key information, inadequate planning, as well as failing to 
re-assess situations as they begin to change, can all lead to mishaps (Naval 
Safety Center, 2007). 

d. Acts 

Acts are shown in Figure 8. Acts are the actions or decisions that 
directly lead to an accident. Acts, or unsafe acts, are categorized within DoD 
HFACS as either errors or violations. 



Figure 8. Categories of unsafe acts 


2. Errors 

Errors come in three forms: Skill-based, judgment and decision making, 
and perception. According to Shappell and Wiegmann, “errors represent the 
mental or physical activities of individuals that fail to achieve their intended 
outcome” (Shappell & Wiegmann, 2001, p. 62). Skill-based errors are generally 
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fairly routine standard activities conducted by individuals. Such errors in skill 
include over-control, not following a checklist or procedure, and unintended 
operation of specific equipment. Errors in judgment and decision making involve 
choosing the wrong course of action in a given situation. These errors can take 
the form of rushing necessary actions, delaying a necessary action, or ignoring 
cautions or warning. Whatever form they take, these errors can nevertheless 
prove costly. Perception errors, or errors due to misperception, occur due to an 
incorrect response to an individual’s senses. 

Violations are the second form of unsafe acts and occur when rules are 
broken or instructions are not followed. Violations occur in the form of work¬ 
around violations, extreme or willful violations by an individual, or even 
widespread and routine violations (Naval Safety Center, 2007). 

3. HFACS Application and Research 

Since its creation, the HFACS taxonomy has been widely used, modified, 
and scrutinized. Due to the high cost of mishaps within industrial and commercial 
sectors, it makes sense for organizations to seek out a system, such as HFACS, 
to classify and count errors better for more effective prevention. However, one 
size does not often fit all. To this end, many researchers have adapted HFACS 
from its original form for use in areas, such as shipboard machinery spaces, the 
mining industry in Australia, and even a version for use with railroad error 
investigation. However, a fair number of skeptics remain who doubt the rating 
consistency (i.e., inter-rater reliability) when HFACS is used in mishap 
investigations. 

More than 80 articles have been published on HFACS since its inception. 
The preponderance of this research presumes HFACS to be a valid, verified 
taxonomy, and use it as such. At the time of their paper in 2001, Shappeli and 
Wiegmann cited inter-rater reliabilities from five studies with an average 
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consensus of between .6 and .95 for a variety of studies. It should be noted that 
these studies were only coded at the categorical level, and not at a nano-code 
level (Shappell & Wiegmann, 2001). 

With such claims of reliability, it is not surprising that many researchers 
accept HFACS at face value. Lenne and his colleagues’ work with safety in the 
Australian mining community used the original 17 categories to code 263 mining 
incidents in Australia from 2007 to 2008 (Lenne, Salmon, Liu, & Trotter, 2011). 
This study used pairs of human factors researchers to translate codes from an 
Incident Cause Analysis Method (IDCAM) model into HFACS codes. Although 
the researchers coded independently, they resolved decision differences in a 
consensus method, much as a SIB would do (Lenne et at., 2011). Studies using 
methods such as these avoid the labor of calculating inter-rater reliability by 
ceding validity to the HFACS model. 

In 2010, Wertheim used HFACS to look at human error in large-scale 
biometric systems. In this research, the use of HFACS was shown to improve 
fingerprint match rate by as much as 10%. Again, however, HFACS was 
assumed to be valid and no inter-rater reliability was not questioned (Wertheim, 
2010 ). 

Like Lenne et al. (2007), the Australian Transport Safety Bureau chose a 
similar path when examining accidents within the Australian civil aviation 
community in 2007. This study again coded accidents using pairs of raters. This 
consensus method is the most common method currently in use among users of 
HFACS according to the research available (Inglis, Sutton, & McRandle, 2007). 

Over the years, new versions of HFACS have been developed. As the 
desire of organizations to narrow down and eliminate causal factors of accidents 
has increased, so has the specificity of HFACS. The preponderance of HFACS 
variants focus on developing systems at the categorical level, with only one 
version daring to venture into the nano-code level. Schroder-Hinrichs and his 
colleagues developed a version of HFACS for machinery spaces on commercial 
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vessels. HFACS-MSS, as it is called, attempts to add specificity to the domain of 
machinery accidents on sea-going vessels. HFACS-MSS adds an additional 
category, outside factors, and changes some of the third tier categories from their 
original form in HFACS to increase specificity (Schroder-Hinrichs, Baldauf, & 
Ghirxi, 2011). 

Similarly, Patterson and Shappell developed HFACS-MI for the Australian 
mining industry. This version, like that of Schroder-Hinrichs’s, adds a fifth 
category of outside factors, which includes regulatory factors that may influence 
future mishaps (Patterson & Shappell, 2010). 

In 2007, Reinach et al. (2007) took HFACS research a step further by both 
developing HFACS-RR for the Federal Railroad Administration and creating a 
software tool to perform much of the work. The Human Error Investigation 
Software Tool was created to effectively “do” HFACS. The program includes 
checklists, guides, a taxonomy, and definitions to assist raters in the process of 
error investigation and identification (Reinach, Viaie, & Green, 2007). 

Despite the bulk of HFACS research being generally positive, skeptics 
remain. It only makes sense that when working for organizations prepared to 
spend millions of dollars to reduce risk and mishaps that studies should focus on 
validating the method of risk identification actually being used. 

O’Connor is perhaps the most well published skeptic of HFACS from an 
inter-rater perspective. He has published three papers examining the reliability of 
HFACS using trained raters and simulating mishap boards. O’Connor’s findings 
demonstrate general unreliability in the usability of HFACS for several reasons. 
O’Connor cites training, experience, and format as possible issues with DoD 
HFACS (O’Connor, 2008; O’Connor, Walliser, & Philips, 2010; O’Connor & 
Walker, 2011). 
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In a study published in 2011, Wang et al. put HFACS to the test using air 
traffic controllers and human factors experts. Using 19 HFACS categories, the 
study showed agreement percentages below 40% for both groups just at the 
categorical level. No testing of nano-codes was conducted (Wang et al., 2011). 

Lastly, in one of the few studies to attempt an adaptation or revision of 
HFACS at the nano-code level, Olsen and Shorrock found results similar to that 
of Wang et al. Their research showed inter-rater reliability at the categorical level 
to be under 50% (Olsen & Shorrock, 2010). 

DoD HFACS is used throughout the U.S. military, as well as organizations 
around the world. It is not, however, a perfect system. Research continues to 
highlight the positive nature of HFACS, but also the negative issues associated 
with its use. 

The largest strength of HFACS lies in its wide applicability and ability to be 
adapted to other uses. One of the best ways to determine the relative usefulness 
of any method is to test it against others that claim to accomplish a similar task. 
Salmon’s research in 2012 compared HFACS with STAMP and Accimap, two 
other systems for error analysis. According to the study, HFACS “lends itself to 
multiple accident case analyses, and so is perhaps more suited to inclusion in 
safety management systems” (Salmon et al., 2012). 

Based on the literature review, the largest strength of HFACS is perhaps 
also the greatest weakness of HFACS. As the system is rather generic, it lacks 
domain specificity, as pointed out by Salmon et al. and Griggs (Salmon et al., 
2012, Griggs, 2012). 

Additionally, while the system is adaptable and able to be transformed 
based on the requirements of the domain, such a process is difficult if the system 
has already been in place. Transforming the resulting codes from hundreds, 
perhaps thousands, of incidents for input into a database would require many 
man-years to re-read incident reports and re-classify each finding. 
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D. THE NEED FOR HFACS MARITIME (HFACS-M) 

The generic nature of DoD HFACS as a one-size-fits-all model is 
insufficient for military components, nearly all of which have domain-specific 
factors associated with them. To improve reliability, the specificity of DoD HFACS 
must improve with regard to the surface Navy. To this end, a maritime version of 
HFACS, HFACS-M, was developed. This version will greatly serve the fleet by 
more accurately and efficiently identifying human error components in accident 
investigation. Additionally, a more fleet-centric version of HFACS will improve 
usability of HFACS and make it more suited for lower category mishaps. Finally, 
domain-specific terminology will reduce the training time required for novices to 
become familiar with HFACS. 

The next chapter describes the development of HFACS-M and the method 
used to test DoD HFACS and HFACS-M. 
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III. METHOD 


A. RESEARCH APPROACH 

This study sought to compare the inter-rater reliability among trained 
raters when using either HFACS or HFACS-M error classification taxonomy to 
code a mishap report. Subjects each received standardized training via a self- 
paced, pre-recorded, voice-over presentation, which provided familiarization with 
the respective taxonomy. Each subject next read through an executive summary 
of a report from the National Transportation Safety Board (NTSB). Subjects were 
asked to review the 11 findings associated with the mishap, and assign 
appropriate codes to each finding based on their understanding of the respective 
taxonomy. Analysis was then conducted to determine the inter-rater reliability 
within each of the two taxonomies, as well as the inter-rater reliability between 
SWOs and non-SWOs. 

B. PARTICIPANTS 

A total of 28 Naval Postgraduate School students, all U.S. military officers 
participated in this study. Gender and age were not determined to be a factor in 
the error classification process and were not recorded. Since DoD HFACS is 
intended for use by all branches of service, no service was excluded from 
participating in the study. Participants included members of the Army, Navy, Air 
Force, Marine Corps, and Coast Guard. Of these participants, five who took the 
case study using DoD HFACS had participated in accident investigations (two 
SWOs and three non-SWOs), and four participants (two SWOs and two non- 
SWOs) using HFACS-M (described in section C.3) had also participated in an 
accident investigation at some point in their careers. None who claimed to have 
participated in an accident investigation had any experience with HFACS in the 
course of those investigations. See Table 1. 
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Table 1. Two-by-two experiment matrix of participants by HFACS version 



DOD HFACS 

HFACS-M 

swo 

7 

7 

NON-SWO 

7 

7 


C. APPARATUS 

This study consisted of three major pieces: self-paced training, a case 
study, and the DoD HFACS and HFACS-M coding sheets. 

1. Training 

The training was conducted via a SAKAI site and featured a series of 
PowerPoint slides with associated voice recording. The presentation offered a 
brief history of either DoD HFACS or HFACS-M, as well as a description of the 
four categories of each of the taxonomies. The latter portion of the presentation 
featured a practice case study with four findings from a fictitious mishap. The 
training divided each of the four findings into its respective category based on the 
taxonomy being employed. Subjects were required to select the nano-code that 
best described the issue stated in the finding. The PowerPoint slides can be 
found in Appendix A. Figure 9 provides the reader with an example of one 
PowerPoint slide and its narration from the DoD HFACS training. 
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V 


Organizational Influences 


• Below are the nano-codes associated with Organizational Influences. 
Take the time now and select the one which best describes the 
following: 

- The parent company of vessel A encouraged a schedule that maximized profits 
and did not allow for proper crew rest 
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Below are the nano-codes associated with Organizational Influences. Take the 
time now and select the one which best describes the following 

The parent company of vessel A encouraged a schedule that did not 
allow for proper crew rest 

Figure 9. Training slide example with speaker notes 

2. Case Study 

The second portion of the apparatus was the case study, which consisted 
of the executive summary of an actual mishap along with the findings from the 
mishap. The mishap was selected from the NTSB database based on its 
moderate number of findings and moderate level of complication. As the NTSB 
has consistent mishap investigation practices, it was determined that in the 
interest of time, it would be well suited for this study. The accident report used in 
this study was NTSB/MAR-11/04, Collision of Tankship Eagle Otome with Cargo 
Vessel Gull Arrow and Subsequent Collision with the Dixie Vengeance Tow. This 
incident occurred in the Sabine-Neches Canal, Port Arthur, Texas, on January 
23, 2010. The executive summary reads as follows. 
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On Saturday, January 23, 2010, about 0935 central standard time, 
the 810-foot-long oil tankship Eagle Otome collided with the 597- 
foot-long general cargo vessel Gull Arrow at the Port of Port Arthur, 
Texas. A 297-foot-long barge, the Kirby 30406, which was being 
pushed by the towboat Dixie Vengeance, subsequently collided 
with the Eagle Otome. The tankship was inbound in the Sabine- 
Neches Canal with a load of crude oil en route to an ExxonMobil 
facility in Beaumont, Texas. Two pilots were on board, as called for 
by local waterway protocol. When the Eagle Otome approached the 
Port of Port Arthur, it experienced several unintended heading 
diversions culminating in the Eagle Otome striking the Gull Arrow, 
which was berthed at the port unloading cargo. 

A short distance upriver from the collision site, the Dixie Vengeance 
was outbound with two barges. The towboat master saw the Eagle 
Otome move toward his side of the canal, and he put his engines 
full astern but could not avoid the subsequent collision. The Kirby 
30406, which was the forward barge pushed by the Dixie 
Vengeance, collided with the Eagle Otome and breached the 
tankship’s starboard ballast tank and the No. 1 center cargo tank a 
few feet above the waterline. As a result of the breach, 862,344 
gallons of oil were released from the cargo tank, and an estimated 
462,000 gallons of that amount spilled into the water. The three 
vessels remained together in the center of the canal while pollution 
response procedures were initiated. No crewmember on board any 
of the three vessels was injured. 

The National Transportation Safety Board (NTSB) determines that 
the probable cause of the collision of tankship Eagle Otome with 
cargo vessel Gull Arrow and the subsequent collision with the Dixie 
Vengeance tow was the failure of the first pilot, who had 
navigational control of the Eagle Otome, to correct the sheering 
motions that began as a result of the late initiation of a turn at a 
mild bend in the waterway. Contributing to the accident was the first 
pilot’s fatigue, caused by his untreated obstructive sleep apnea and 
his work schedule, which did not permit adequate sleep; his 
distraction from conducting a radio call, which the second pilot 
should have conducted in accordance with guidelines; and the lack 
of effective bridge resource management by both pilots. Also 
contributing was the lack of oversight by the Jefferson and Orange 
County Board of Pilot Commissioners. 
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Following the executive summary was a partial list of findings from the 
accident investigation presented to the participants. They read as follows. 

Based on your knowledge of the associated error classification 
taxonomy and your understanding of the facts surrounding the 
investigation, assign an appropriate nano-code that best describes 
each of the findings listed below. Please note that there is no right 
or wrong answer. Carefully read and consider the possible options 
before answering. 

1. The Eagle Otome pilots did not follow Sabine Pilots 
Association guidelines with respect to division of duties while 
under way. 

2. Although both pilots completed bridge resource 
management training, they failed to apply the team 
performance aspects of bridge resource management to this 
operation. 

3. Contrary to pilot association guidelines, the first pilot on the 
Eagle Otome was conducting a radio call at a critical point in 
the waterway, and the radio call interfered with his ability to 
fully focus on conning the vessel. 

4. Had the Eagle Otome pilots alerted the Dixie Vengeance 
master of the sheering problem, the force of the collision 
between the Eagle Otome and the Dixie Vengeance tow 
would have been lessened or the collision might have been 
avoided altogether. 

5. The combination of untreated obstructive sleep apnea, 
disruption to his circadian rhythms, and extended periods of 
wakefulness that resulted from his work schedule caused the 
first pilot to be fatigued at the time of the accident. 

6. The first pilot’s failure to correct the sheering motions that 
began after his late turn initiation at Missouri Bend led to the 
accident. 

7. The first pilot’s fatigue adversely affected his ability to predict 
and stop the Eagle Otome’s sheering. 

8. No effective hours of service rules were in place that would 
have prevented the Sabine pilots from being fatigued by the 
schedules that they maintained. 
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9. The absence of an effective fatigue mitigation and 
prevention program among the pilots operating under the 
authority of the Jefferson and Orange County Board of Pilot 
Commissioners created a threat to the safety of the 
waterway, its users, and those nearby. 

10. The Jefferson and Orange County Board of Pilot 
Commissioners should have more fully exercised its 
authority over pilot operations on the Sabine-Neches 
Waterway by becoming aware of and enforcing the Sabine 
Pilots Association’s two-pilot guidelines and implementing a 
fatigue mitigation and prevention program among the Sabine 
pilots. 

11. Commonly accepted human factors principles were not 
applied to the design of the Eagle Otome’s engine control 
console, which increased the likelihood of error in the use of 
the controls. 

The following findings from the mishap investigation were not presented to the 
participants because either they did not actually address an error or they 
speculated on or made recommendations for future improvements. 

• Weather, mechanical failure, and illegal drug or alcohol use were 
not factors in the accident. 

• The vessel meeting arrangement agreed to by the towboat master 
and the first pilot was appropriate and was not a factor in the 
accident. 

• Personnel at Vessel Traffic Service Port Arthur played no role in the 
accident. 

• The Coast Guard is the organization with the resources, 
capabilities, and expertise best suited to (1) enhance 
communication among pilot oversight organizations and 
(2) establish an easy-to-use and readily available database of pilot 
incidents and accidents. 

• The first pilot’s sounding the Eagle Otome’s whistle and the Gull 
Arrow master’s sounding the cargo vessel’s general alarm were 
prudent and effective. 

• The accident response and oil spill recovery efforts were timely and 
effective. 
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• The dimensions of the Sabine-Neches Waterway may pose an 
unacceptable risk, given the size and number of vessels transiting 
the waterway. 

• Consistent use of a vessel’s name in radio communication can help 
avoid confusion and enhance bridge team coordination 

3. DoD HFACS and HFACS-M 

Participants received training on either DoD HFACS or HFACS-M, and 
received corresponding coding sheets. The categories, sub-codes, and nano¬ 
codes used in the DoD HFACS coding sheets were taken directly from the Naval 
Safety Center’s 2007 booklet, “DoD Human Factors Analysis and Classification 
System (HFACS).” 

The coding sheet was divided by category, sub-code, and nano-code as 
shown in Figure 10. Each nano-code was given its own row of 11 boxes 
representing the 11 findings of the accident investigation. 
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DOD HFACS Findings 


Naval Safety Center, 2007 version _ Mark an X in the box below associated with your choice for the best 


ACTS 

fit nanocode for each of the findings 

Nano-Code 

Description 


Skill-Based Errors 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

AE 101 

Unintended operation of equipment 












AE 102 

Checklist not followed correctly 












AE 103 

Procedure not followed correctly 












AE 104 

Over-Control led/under-control led aircraft/vehicle 












AE 105 

Breakdown in visual scan 












AE 106 

Inadequate Anti-G straining maneuver 












Judgement and Decision-Making Errors 












AE 201 

Inadequate real-time risk assessment (e.g., failure of time-critical ORM) 












AE 202 

Failure to prioritize tasks adequately 












AE 203 

Rushed a necessary action 












AE 204 

Delayed a necessary action 












AE 205 

Ignored a caution/warning 












AE 206 

Wrong choice of action during an operation (e.g., response to an emergency) 












Perception Errors 












AE 301 

Incorrect response to a misperception (e.g., visual illusion or spatial disorientation) 












Violations 












AV 001 

Work-around violation (e.g., breaking the rules is prceived as the best solution) 












AV 002 

Widespread/routine violation (e.g., habitual deviation from the rules that is tolerated by management) 












AV 003 

Extreme violation (e.g., a violation not condoned by management 













Figure 10. DoD HFACS coding sheet example 
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HFACS-M was presented in the same manner as DoD HFACS. HFACS-M 
was created by modifying the original 2007 version of DoD HFACS to make it 
more specific to the surface Navy. To this end, the following modifications were 
made. 


• AE102—Rephrased—Checklist not followed/not followed correctly 

• AE 103—Rephrased—Procedure not followed/not followed 
correctly 

• AE 104—Rephrased—Over-Controlled or under-controlled vessel 

• AE 106—Removed (N/A for shipboard use)—Inadequate Anti-G 
straining maneuver 

• PE 101—Rephrased—Icing/fog on window restricts visibility 

• PE 102—Rephrased—Weather conditions restrict visibility 

• PE 103—Rephrased—Vibrations/rolls affect vision or balance 

• PEI 09—Rephrased—Backlighting/backscatter interfere with 
performance 

• PE112—Added—High winds/Heavy seas affect/impair movement 

• PE201—Removed (N/A for shipboard use)—Seat and restraint 
systems problems 

• PE208—Added—Equipment not configured correctly 

• PE209—Added—Corrective maintenance not conducted/not 
conducted correctly 

• PE210—Added—Preventive maintenance not conducted/not 
conducted correctly 

• PP101 —Rephrased—Failure of watchteam/crew leadership 

• PC301—Removed (N/A for shipboard use)—Effects of G forces 
(e.g., G-LOC) 

• PC 304—Removed N/A parenthesis—Sudden 
incapacitation/unconsciousness (not due to G) 

• PC 308—Rephrased—Circadian rhythm de-synchronization (watch 
rotation or shift work) 

• PC 310—Removed (N/A for shipboard use)—Trapped gas 
disorders 

• PC311—Removed (N/A for shipboard use)—Evolved gas disorders 
(e.g., decompression sickness/bends 
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Supervision—Renamed—Command 

• SI007—Added—Failed to communicate intent (e.g., standing 
orders/night orders) 

Manning/Personnel/Training Issues—Added new subcategory 

• SP007—Added—Directed mission without sufficient manning 

• SP007—Command (formal) training is inadequate 

• SP008—Rephrased—Performed inadequate risk assessment 
(ORM) 

• SV004—Moved to MPT sub-category 

• OROOI—Rephrased—Port facilities are deficient 

• OR002—Channel markers/lighting are deficient 

• OR005—Added—Failure to procure new systems/upgrades in a 
timely manner 

• OP007—Organizational process provides inadequate, untimely 
guidance 

These changes were necessary to remove ambiguity and to fill gaps in DoD 
FI FACS because of the generic nature of the taxonomy. 

The coding sheet for FIFACS-M was also divided by category, sub-code, 
and nano-code as shown in Figure 11. Again, each nano-code was given its own 
row of 11 boxes representing the 11 findings of the accident investigation. 
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HFACS-M Findings 


Naval Postgraduate School, 2013 version _ Mark an X in the box below associated with your choice for the best 


ACTS 

fit nanocode for each of the findings 

Nano-Code 

Descri ption 


Skill-Based Errors 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

AE 101 

Unintended operation of equipment 












AE 102 

Checklist not followed/not followed correctly 












AE 103 

Procedure not followed/not followed correctly 












AE 104 

Over-Controlled or under-controlled vessel 












AE 105 

Breakdown in visual scan 












Judgement and Decision-Making Errors 












AE 201 

Inadequate real-time risk assessment (e.g., failure of time-critical ORM) 












AE 202 

Failure to prioritize tasks adequately 












AE 203 

Rushed a necessary action 












AE 204 

Delayed a necessary action 












AE 205 

Ignored a caution/warning 












AE 206 

Wrong choice of action during an operation (e.g., response to an emergency) 












Perception Errors 












AE 301 

Incorrect response to a misperception (e.g., visual illusion or spatial disorientation) 












Violations 












AV 001 

Work-around violation (e.g., breaking the rules is prceived as the best solution) 












AV 002 

Widespread/routine violation (e.g., habitual deviation from the rules that is tolerated by management) 












AV 003 

Extreme violation (e.g., a violation not condoned by management 













Figure 11. HFACS-M coding sheet example 


39 








































D. PROCEDURES 

The Naval Postgraduate School’s Institutional Review Board (IRB) 
reviewed and approved this research. Volunteers were recruited via email from 
the student body. They reported to the Human Systems Integration Laboratory 
and were met by the student researcher. They were asked to sit in front of a 
computer with either the DoD HFACS or HFACS-M training loaded on it. The 
subjects read and signed the informed consent form before proceeding. Next, 
each subject viewed the voice-recorded training slides. Subjects were instructed 
to progress through the slides at their own pace. Upon reaching the practice 
slides, subjects were instructed to read through all the possible nano-codes 
before making a selection. They were given a pen and scratch paper with which 
to take notes as desired. 

Upon completion of the training, each subject was asked to answer the 
following questions. 

1. Have you completed the associated training? Yes No 

2. Have you ever been involved in an accident investigation? Yes No 

3. Have you ever used HFACS in the course of an accident investigation? Yes No 

4. What is your current designator/MOS/AFSC? _ 

Next, the subjects were instructed to read the executive summary from the 
NTSB accident report. Following this, they were given the list of 11 findings from 
the accident report and asked to assign one and only one nano-code from the 
taxonomy they were given that, in their judgment, best described the finding. 
Once the subjects finished marking all their selections, they were debriefed and 
thanked for their assistance. 
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E. DATA ANALYSIS 

Upon completion of data collection, it was determined that no respondent 
data would be excluded. None of the subjects had used HFACS previously. 
Although several had been involved in accident investigations, it was determined 
by the research team that the experience did not give them any significant 
advantage. 

The tables completed by individual raters were compiled into a data table. 
A Fleiss’ Kappa analysis was conducted to determine the inter-rater reliability of 
those subjects using DoD HFACS compared to those who coded using HFACS- 
M. A Fleiss’ Kappa analysis was also conducted to determine the inter-rater 
reliability between SWOs (maritime domain experts), and non-SWOs. These 
analyses were conducted at the categorical, sub-code, and nano-code levels. 
Fleiss’ Kappa was used to determine inter-rater reliability among multiple raters, 
rather than Cohen’s Kappa, which is designed for only two raters (Fleiss, 1971). 
Following the determination of Fleiss’ Kappa for each data set, a simulation was 
conducted in R to determine the significance of the findings. See Fleiss (1971) 
for a description and explanation of Fleiss’ Kappa. 
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IV. RESULTS 


A. DESCRIPTION OF PARTICIPANTS 

Twenty-eight Naval Postgraduate School students took part in this study. 
Subjects included members from each branch of service. Students self-identified 
their MOS/AFSC/Designator in the questionnaire provided. Table 1 shows the 
breakdown of participants. All told, 14 SWOs and 14 non-SWOS participated in 
the study. Participants were alternated between versions of HFACS. 

B. NANO-CODE ANALYSIS 

Each participant selected one nano-code from either DoD HFACS or 
HFACS-M for each of the 11 findings in the NTSB investigation. These selections 
were compiled into two tables, one for DoD HFACS and one for HFACS-M. 
Participants 1-7 of Table 2 and Table 3 were non-SWOs and participants 8-14 
were SWOs. 
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Table 2. DoD HFACS results broken down by Designator/MOS/AFSC 


DOD HFACS 

Findings 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

49/15 

AV001 

PP103 

AE202 

PP106 

PC307 

AE104 

PC307 

OP003 

OP006 

OP002 

PE204 

2 

1310 

PP103 

PP101 

PC108 

PP106 

PC307 

AE204 

PC307 

OPOOl 

OCOOl 

SI001 

PP110 

3 

0602 

AE103 

AE103 

AE206 

PP106 

PC307 

AE206 

PC307 

SI004 

SI004 

SF001 

PE204 

4 

7565 

AE103 

PP101 

AE202 

PP106 

PC307 

AE204 

PC504 

SV002 

SI003 

SF002 

PE204 

5 

49 A 

PP103 

OP004 

PC102 

PP106 

PP205 

PC504 

PC404 

OP002 

OP003 

SI 004 

PE206 

6 

21B/49 

AE202 

AV001 

PC106 

PP106 

SI001 

AE204 

PP205 

OR007 

OP005 

OP006 

OR004 

7 

1810 

PP103 

PP101 

PC106 

PP106 

PC307 

AE103 

AE201 

OCOOl 

SI004 

SI001 

PE204 

8 

1110 

AE103 

OP004 

PC108 

AE204 

PC308 

AE104 

PC308 

SI004 

SF001 

SF002 

PE204 

9 

1110 

SV001 

AE103 

AV001 

AE206 

OPOOl 

AE206 

OPOOl 

OP002 

OP005 

OP002 

OR004 

10 

1110 

SV002 

OP004 

AE103 

PP108 

PC308 

AE206 

PC307 

SI001 

OP002 

OP006 

PE204 

11 

1110 

AE103 

PP103 

PC106 

PP106 

PC308 

AE206 

OCOOl 

OCOOl 

OCOOl 

OP005 

PE204 

12 

1110 

AV001 

AE202 

AE202 

AE204 

PC307 

PP111 

PP205 

SI004 

SF002 

SI001 

PE204 

13 

1110 

AV003 

PP101 

AE206 

PP106 

PC305 

AE104 

PC307 

OP002 

OP003 

OP006 

PE204 

14 

1110 

AE202 

PP102 

SF001 

PP106 

PP206 

AE104 

PC307 

SF001 

OP002 

OP006 

PE207 


Table 3. HFACS-M results broken down by Designator/MOS/AFSC 


HFACS-M 

Findings 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

1317 

AE103 

AE201 

AE201 

PP106 

PC306 

AE104 

PC306 

SI004 

SF002 

SI001 

PE203 

2 

1810 

AE103 

PP112 

PC106 

PP106 

PC304 

AE204 

PC306 

SP201 

OP006 

OP002 

PE203 

3 

0602 

AV002 

SI001 

PC106 

AE204 

PC304 

PC101 

PP205 

OPOOl 

OP002 

OP006 

OR003 

4 

7523 

AE201 

AE103 

PP108 

AE206 

PC307 

AE204 

PC306 

PE204 

PE202 

SI001 

OR004 

5 

1310 

AE103 

PP101 

AE202 

PP106 

PC306 

AE206 

PC306 

OP003 

SI004 

SI006 

OR004 

6 

1120 

AV001 

PP102 

AE203 

PP106 

OPOOl 

AE206 

PC505 

OPOOl 

OR007 

OP006 

PE206 

7 

19A 

AV001 

PP102 

PP101 

PP106 

PC307 

AE204 

PC306 

SF001 

OP006 

SV002 

PE201 

8 

1110 

AV001 

PC206 

PP103 

PP106 

PC304 

AE204 

PC307 

PC 307 

OP002 

OCOOl 

PE201 

9 

1110 

AE103 

AE102 

PC108 

PP106 

PC307 

PP105 

PC307 

OPOOl 

OP002 

OP006 

PE203 

10 

1110 

AE103 

PP101 

PC108 

PP106 

PC306 

AE104 

PC510 

SI004 

SF002 

SI001 

PE208 

11 

1110 

AV001 

PC405 

PC102 

PP106 

PC307 

AE204 

PC306 

OP003 

OP005 

OCOOl 

OR004 

12 

1110 

AE103 

AE206 

AE206 

AE204 

OPOOl 

AE204 

PC306 

OP003 

OCOOl 

OP007 

PE203 

13 

1110 

AE103 

PP101 

PC108 

PP106 

PC306 

AE204 

PC306 

OP002 

OP005 

SI 004 

OP006 

14 

1110 

PC306 

PP101 

PP108 

PP106 

PC306 

AE204 

PC307 

OCOOl 

SP007 

SI001 

OR004 
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1 . 


DoD HFACS 


From these results, tables were constructed to calculate Fleiss’ Kappa. 
Table 4 shows an example. Each nano-code was assigned its own column and 
Pj, the proportion of assignments that were to the y-th category, was calculated 

for each. The rows delineate the finding with which the code is associated. In 
DoD HFACS, 147 possible nano-codes were available. 


Table 4. DoD HFACS nano-code table example 


DODNANO 

AE101 

AE102 

AE103 

AE104 

AE105 

AE106 

1 

0 

0 

4 

0 

0 

0 

2 

0 

0 

2 

0 

0 

0 

3 

0 

0 

1 

0 

0 

0 

4 

0 

0 

0 

0 

0 

0 

5 

0 

0 

0 

0 

0 

0 

6 

0 

0 

1 

4 

0 

0 

7 

0 

0 

0 

0 

0 

0 

8 

0 

0 

0 

0 

0 

0 

9 

0 

0 

0 

0 

0 

0 

10 

0 

0 

0 

0 

0 

0 

11 

0 

0 

0 

0 

0 

0 

Total 

0 

0 

8 

4 

0 

0 

Pj 

0.000 

0.000 

0.052 

0.026 

0.000 

0.000 


Fleiss’ Kappa was calculated to determine inter-rater reliability at the 
nano-code level for raters using DoD HFACS. Fleiss’ Kappa was derived by first 
taking the difference of P and Pe to find the degree above chance that was 
achieved. This difference is then divided by 1 -Pe to obtain Fleiss’ Kappa. The 
overall results were as follows: P = .185; Pe = .036; K = .154. 

The results were then divided between SWOs and non-SWOs. Fleiss’ 
Kappa was calculated for each group individually. For SWOs employing DoD 
HFACS at the nano-code level, the results were as follows: P = .147; Pe = .038; 
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K = .114. For non-SWOs employing DoD HFACS at the nano-code level, the 
results were as follows: P = .234; Pe = .045; K = .198. 

2. HFACS-M 

Fleiss’ Kappa was also calculated to determine inter-rater reliability at the 
nano-code level for raters using FIFACS-M. In HFACS-M, 152 possible nano¬ 
codes were available. For HFACS-M at the nano-code level, the overall results 
were as follows: P = .212; Pe = .037; ^=.182. 

The results were then divided between SWOs and non-SWOs. Fleiss’ 
Kappa was calculated for each group individually. For SWOs employing HFACS- 
M at the nano-code level, the results were as follows: P = .238; Pe= .046; K = 
.202 For non-SWOs employing HFACS-M at the nano-code level, the results 
were as follows: P = .169; Pe = .037; K = .137. 

C. SUB-CODE LEVEL 

Each of the nano-codes in the DoD HFACS and HFACS-M taxonomies 
falls under a specific sub-code. For this next level of analysis, the nano-codes 
were translated into their respective sub-code within the original tables. 
Participants 1-7 of Table 5 and Table 6 were non-SWOs and participants 8-14 
were SWOs. 


46 



Table 5. DoD HFACS sub-codes broken down by Designator/MOS/AFSC 


DOD HFACS 

Findings 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

49/15 

A-V 

P-CCPF 

A-JDME 

P-CCPF 

P-APS 

A-SB 

P-APS 

O-P 

O-P 

O-P 

P-TE 

2 

1310 

P-CCPF 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

O-C 

S-IS 

P-CCPF 

3 

0602 

A-SB 

A-SB 

A-JDME 

P-CCPF 

P-APS 

A-JDME 

P-APS 

S-IS 

S-IS 

S-FCKP 

P-TE 

4 

7565 

A-SB 

P-CCPF 

A-JDME 

P-CCPF 

P-APS 

A-JDME 

P-PF 

S-SV 

S-IS 

S-FCKP 

P-TE 

5 

49A 

P-CCPF 

O-P 

P-AF 

P-CCPF 

P-SIS 

P-PF 

P-PML 

O-P 

O-P 

S-IS 

P-TE 

6 

21B/49 

A-JDME 

A-V 

P-AF 

P-CCPF 

S-IS 

A-JDME 

P-SIS 

O-RAM 

O-P 

O-P 

O-RAM 

7 

1810 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-SB 

A-JDME 

O-C 

S-IS 

S-IS 

P-TE 

8 

1110 

A-SB 

O-P 

P-AF 

A-JDME 

P-APS 

A-SB 

P-APS 

S-IS 

S-FCKP 

S-FCKP 

P-TE 

9 

1110 

S-SV 

A-SB 

A-V 

A-JDME 

O-P 

A-JDME 

O-P 

O-P 

O-P 

O-P 

O-RAM 

10 

1110 

S-SV 

O-P 

A-SB 

P-CCPF 

P-APS 

A-JDME 

P-APS 

S-IS 

O-P 

O-P 

P-TE 

11 

1110 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-JDME 

O-C 

O-C 

O-C 

O-P 

P-TE 

12 

1110 

A-V 

A-JDME 

A-JDME 

A-JDME 

P-APS 

P-CCPF 

P-SIS 

S-IS 

S-FCKP 

S-IS 

P-TE 

13 

1110 

A-V 

P-CCPF 

A-JDME 

P-CCPF 

P-APS 

A-SB 

P-APS 

O-P 

O-P 

O-P 

P-TE 

14 

1110 

A-JDME 

P-CCPF 

S-FCKP 

P-CCPF 

P-SIS 

A-SB 

P-APS 

S-FCKP 

O-P 

O-P 

P-TE 
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Table 6. HFACS-M sub-codes broken down by Designator/MOS/AFSC 


HFACS-M 


Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

1317 

A-SB 

A-JDME 

A-JDME 

P-CCPF 

P-APS 

A-SB 

P-APS 

C-IS 

C-FCKP 

C-IS 

P-TE 

2 

1810 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

C-PIO 

O-P 

O-P 

P-TE 

3 

0602 

A-V 

C-IS 

P-AF 

A-JDME 

P-APS 

P-AF 

P-SIS 

O-P 

O-P 

O-P 

O-RAM 

4 

7523 

A-JDME 

A-SB 

P-CCPF 

A-JDME 

P-APS 

A-JDME 

P-APS 

P-TE 

P-TE 

C-IS 

O-RAM 

5 

1310 

A-SB 

P-CCPF 

A-JDME 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

C-IS 

C-IS 

O-RAM 

6 

1120 

A-V 

P-CCPF 

A-JDME 

P-CCPF 

O-P 

A-JDME 

P-PF 

O-P 

O-RAM 

O-P 

P-TE 

7 

19A 

A-V 

P-CCPF 

P-CCPF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

C-FCKP 

O-P 

C-SV 

P-TE 

8 

1110 

A-V 

P-PBF 

P-CCPF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

O-P 

O-C 

P-TE 

9 

1110 

A-SB 

A-SB 

P-AF 

P-CCPF 

P-APS 

P-CCPF 

P-APS 

O-P 

O-P 

O-P 

P-TE 

10 

1110 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-SB 

P-PF 

C-IS 

C-FCKP 

C-IS 

P-TE 

11 

1110 

A-V 

P-PML 

P-AF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

O-P 

O-C 

O-RAM 

12 

1110 

A-SB 

A-JDME 

A-JDME 

A-JDME 

O-P 

A-JDME 

P-APS 

O-P 

O-C 

O-P 

P-TE 

13 

1110 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

O-P 

C-IS 

O-P 

14 

1110 

P-APS 

P-CCPF 

P-CCPF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-C 

C-MPT 

C-IS 

O-RAM 
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1 . 


DoD HFACS 


From these results, tables were constructed to calculate Fleiss’ Kappa at 
the sub-code level. Each sub-code was once again assigned its own column and 
Pj was calculated for each. In DoD HFACS, 20 possible sub-codes were 

available. Table 7 shows the overall breakdown of sub-codes, shown in the 
columns, and findings, represented by the rows. 


49 



Table 7. Overall DoD HFACS sub-code table 


DODSUB 

A-SB 

A-JDME 

A-PE 

A-V 

P-PE 

P-TE 

P-SIS 

P-CCPF 

P-AF 

P-PML 

P-PF 

P-PBF 

P-APS 

S-iS 

S-FCKP 

S-PIO 

s-sv 

O-RAM 

o-c 

O-P 

Pi 

1 

5 

2 

0 

3 

0 

0 

0 

2 

0 

0 

0 

0 

0 

0 

0 

0 

2 

0 

0 

0 

0.176 

2 

2 

1 

0 

1 

0 

0 

0 

7 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3 

0.275 

3 

1 

5 

0 

1 

0 

0 

0 

0 

6 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0.275 

4 

0 

3 

0 

0 

0 

0 

0 

11 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.637 

5 

0 

0 

0 

0 

0 

0 

2 

0 

0 

0 

0 

0 

10 

1 

0 

0 

0 

0 

0 

1 

0.505 

6 

5 

7 

0 

0 

0 

0 

0 

1 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.341 

7 

0 

1 

0 

0 

0 

0 

2 

0 

0 

1 

1 

0 

7 

0 

0 

0 

0 

0 

1 

1 

0.242 

8 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

4 

1 

0 

1 

1 

2 

5 

0.187 

9 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3 

2 

0 

0 

0 

2 

7 

0.286 

10 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

4 

3 

0 

0 

0 

0 

7 

0.330 

11 

0 

0 

0 

0 

0 

11 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

2 

0 

0 

0.615 

Total 

13 

19 

0 

5 

0 

11 

4 

22 

6 

1 

2 

0 

17 

12 


0 

3 

3 

3 

24 

3.868 


[Pj_ | 0.084 0.123 0.000 0.032 0.000 0.071 0.026 0.143 0.039 0.006 0.013 0.000 0.110 0.078 0.045 0.000 0.019 0.019 0.032 0.156 
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Fleiss’ Kappa was calculated to determine inter-rater reliability at the sub¬ 
code level for raters using DoD HFACS. The results were as follows: P = .352; 
Pe= .098; K = .281. 

Table 7 was then divided between SWOs and non-SWOs. Fleiss’ Kappa 
was calculated for each group individually. For SWOs employing DoD FIFACS at 
the sub-code level, the results were as follows: P = .329; Pe = .106; K = .250 
For non-SWOs employing DoD HFACS at the sub-code level, the results were as 
follows: P = .364; Pe = .099; K = .293. 

2. HFACS-M 

Fleiss’ Kappa was also calculated to determine inter-rater reliability at the 
sub-code level for raters using HFACS-M. In HFACS-M, 21 possible sub-codes 
were available. Table 8 shows the overall breakdown of sub-codes, shown in the 
columns, and findings, represented by the rows. 
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Table 8. HFACS-M sub-code table 


HFACS-M SUB 

A-SB 

A-JDME 

A-PE 

A-V 

P-PE 

P-TE 

P-SIS | 

P-CCPF 

P-AF 

P-PML 

P-PF 

P-PBF 

P-APS 

C-IS 

C-FCKP 

C-MPT 

C-PIO 

c-sv 

O-RAM 

o-c 

O-P 

Pi 

1 

7 

1 

0 

5 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0.341 

2 

2 

2 

0 

0 

0 

0 

0 

7 

0 

1 

0 

1 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0.253 

3 

0 

4 

0 

0 

0 

0 

0 

4 

6 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.297 

4 

0 

3 

0 

0 

0 

0 

0 

11 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.637 

5 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

12 

0 

0 

0 

0 

0 

0 

0 

2 

0.736 

6 

2 

10 

0 

0 

0 

0 

0 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.505 

7 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

2 

0 

11 

0 

0 

0 

0 

0 

0 

0 

0 

0.615 

8 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

2 

1 

0 

1 

0 

0 

1 

8 

0.319 

9 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

1 

2 

1 

0 

0 

1 

1 

7 

0.242 

10 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

6 

0 

0 

0 

1 

0 

2 

5 

0.286 

11 

0 

0 

0 

0 

0 

8 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5 

0 

1 

0.418 

Total 

11 

20 

° 

5 

_°L 

10 

_ A 

23 

7 

_iL 

2 

1 

_ 

10 

3 

1 

1 

1 

6 


23 

4.648 


Pj 0.071 0.130 0.000 0.032 0.000 0.065 0.006 0.149 0.045 0.006 0.013 0.006 0.156 0.065 0.019 0.006 0.006 0.006 0.039 0.026 0.149 
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Fleiss’ Kappa was calculated to determine inter-rater reliability at the 
sub-code level for raters using HFACS-M. The overall results were as follows: 
P = .423; Pe = .105; K = .355. 

Table 8 was then divided between SWOs and non-SWOs. Fleiss’ Kappa 
was calculated for each group individually. For SWOs employing FIFACS-M at 
the sub-code level, the results were as follows: P = .433; Pe = .111; K = .362 
For non-SWOs employing HFACS-M at the sub-code level, the results were as 
follows: P = .364; Pe = .105; K = .289. 

D. CATEGORICAL LEVEL 

Each of the sub-codes in the DoD HFACS and HFACS-M taxonomies falls 
under a given category. For this next level of analysis, the sub-codes were 
translated into their respective category within the original tables. In DoD HFACS 
these categories consisted of organizational influences, supervision, 
preconditions, and acts. HFACS-M changes the supervision category to 
command. Participants 1-7 of Table 9 and Table 10 were non-SWOs and 
participants 8-14 were SWOs. 
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Table 9. DoD HFACS categories broken down by Designator/MOS/AFSC 


DOD HFACS 

Findings 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

49/15 

A 

P 

A 

P 

P 

A 

P 

O 

O 

0 

p 

2 

1310 

P 

P 

P 

P 

P 

A 

P 

O 

O 

0 

p 

3 

0602 

A 

A 

A 

P 

P 

A 

P 

O 

O 

0 

p 

4 

7565 

A 

P 

A 

P 

P 

A 

P 

S 

S 

s 

p 

5 

49A 

P 

O 

P 

P 

P 

P 

P 

O 

O 

s 

p 

6 

21B/49 

A 

A 

P 

P 

S 

A 

P 

O 

O 

0 

0 

7 

1810 

A 

P 

P 

P 

P 

A 

A 

O 

S 

s 

p 

8 

1110 

A 

O 

P 

A 

P 

A 

P 

S 

S 

s 

p 

9 

1110 

S 

A 

A 

A 

O 

A 

O 

O 

O 

0 

0 

10 

1110 

S 

O 

A 

P 

P 

A 

P 

S 

O 

0 

p 

11 

1110 

A 

P 

P 

P 

P 

A 

O 

O 

O 

0 

p 

12 

1110 

A 

A 

A 

A 

P 

P 

P 

S 

S 

s 

p 

13 

1110 

A 

P 

A 

P 

P 

A 

P 

O 

O 

0 

p 

14 

1110 

A 

P 

S 

P 

P 

A 

P 

S 

0 

0 

p 
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Table 10. HFACS-M categories broken down by Designator/MOS/AFSC 


HFACS-M 


Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

1317 

A 

A 

A 

P 

P 

A 

P 

C 

C 

c 

p 

2 

1810 

A 

A 

P 

P 

P 

A 

P 

C 

O 

0 

p 

3 

0602 

A 

C 

P 

A 

P 

P 

P 

O 

O 

0 

0 

4 

7523 

A 

A 

P 

A 

P 

A 

P 

P 

P 

c 

0 

5 

1310 

A 

P 

A 

P 

P 

A 

P 

O 

C 

c 

0 

6 

1120 

A 

P 

A 

P 

O 

A 

P 

O 

O 

0 

p 

7 

19A 

A 

P 

P 

P 

P 

A 

P 

C 

O 

c 

p 

8 

1110 

A 

P 

P 

P 

P 

A 

P 

O 

O 

0 

p 

9 

1110 

A 

A 

P 

P 

P 

P 

P 

O 

O 

0 

p 

10 

1110 

A 

A 

P 

P 

P 

A 

P 

C 

C 

c 

p 

11 

1110 

A 

P 

P 

P 

P 

A 

P 

O 

0 

0 

0 

12 

1110 

A 

A 

A 

A 

O 

A 

P 

O 

0 

0 

p 

13 

1110 

A 

A 

P 

P 

P 

A 

P 

O 

0 

c 

0 

14 

1110 

P 

P 

P 

P 

P 

A 

P 

O 

c 

c 

0 
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1 . 


DoD HFACS 


From these results, tables were constructed to calculate Fleiss’ Kappa. 
Each category was assigned its own column and P. was calculated for each. 

Four possible categories were available in DoD HFACS. Table 11 shows the 
overall breakdown of categories, shown in the columns, and findings, 
represented by the rows. 


Table 11. Overall DoD HFACS category table 


DOD CAT 

O 

S 

P 

A 

Pi 

1 

0 

2 

2 

10 

0.516 

2 

3 

0 

7 

4 

0.330 

3 

0 

1 

6 

7 

0.396 

4 

0 

0 

11 

3 

0.637 

5 

1 

1 

12 

0 

0.725 

6 

0 

0 

2 

12 

0.736 

7 

2 

0 

11 

1 

0.615 

8 

9 

5 

0 

0 

0.505 

9 

10 

4 

0 

0 

0.560 

10 

9 

5 

0 

0 

0.505 

11 

2 

0 

12 

0 

0.736 

Total 

36 

18 

63 

37 

6.264 


Pj 0.234 0.117 0.409 0.240 


Fleiss’ Kappa was calculated to determine inter-rater reliability at the 
categorical level for raters using DoD HFACS. The overall results were as 
follows: P = .569; Pe = .293; K = .391. 

Table 11 was then divided between SWOs and non-SWOs. Fleiss’ Kappa 
was calculated for each group individually. For SWOs employing DoD HFACS at 
the category level, the results were as follows: P = .515; Pe = .272; K = .334. 
For non-SWOs employing DoD HFACS at the category level, the results were as 
follows: P = .619; Pe = .324; K = .436. 
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2 . 


HFACS-M 


Fleiss’ Kappa was also calculated to determine inter-rater reliability at the 
categorical level for raters using HFACS-M. Four possible categories were 
available. Table 12 shows the overall breakdown of categories, shown in the 
columns, and findings, represented by the rows. 


Table 12. HFACS-M category table 


M CAT 

O 

C 

P 

A 

Pi 

1 

0 

0 

1 

13 

0.857 

2 

0 

1 

6 

7 

0.396 

3 

0 

0 

10 

4 

0.560 

4 

0 

0 

11 

3 

0.637 

5 

2 

0 

12 

0 

0.736 

6 

0 

0 

2 

12 

0.736 

7 

0 

0 

14 

0 

1.000 

8 

9 

4 

1 

0 

0.462 

9 

9 

4 

1 

0 

0.462 

10 

7 

7 

0 

0 

0.462 

11 

6 

0 

8 

0 

0.473 

Total 

33 

16 

66 

39 

6.780 


Pj 0.214 0.104 0.429 0.253 


Fleiss’ Kappa was calculated to determine inter-rater reliability at the 
categorical level for raters using DoD HFACS. The overall results were as 
follows: P = .616; Pe = .305; K = .448. 

Table 12 was then divided between SWOs and non-SWOs. Fleiss’ Kappa 
was calculated for each group individually. For SWOs employing HFACS-M at 
the category level, the results were as follows: P = .645; Pe = .317; K = .481. 
For non-SWOs employing DoD HFACS at the category level, the results were as 
follows: P = .558; Pe = .297; K = .372. 
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Table 13 shows a side-by-side comparison of Fleiss’ Kappa for both 
HFACS versions at each of the three levels analyzed. 


Table 13. Fleiss’ Kappa comparison of DoD HFACS and 
HFACS-M results at all three levels 



Overall 

SWO 

NON-SWO 

DOD HFACS (Nano-code) 

0.154 

0.114 

0.198 

HFACS-M (Nano-code) 

0.182 

0.202 

0.137 

DOD HFACS (Sub-code) 

0.281 

0.25 

0.293 

HFACS-M (Sub-code) 

0.355 

0.362 

0.289 

DOD HFACS (Category) 

0.391 

0.334 

0.436 

HFACS-M (Category) 

0.448 

0.481 

0.372 


The following 


chapter discusses the results of this 


analysis and their 


implications. 
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V. DISCUSSION 


A. DISCUSSION 

Accident investigations have concluded that virtually all major mishaps 
that occurred within the surface Navy are the product of human error (Lacy, 
1998). To mitigate or prevent mishaps of this nature, it is vital that an appropriate 
method be established to categorize and count these errors. DoD HFACS is one 
method that has been employed for several years, but its reliability has been 
called into question on more than one occasion. To this end, a domain-specific 
version, HFACS-M, was developed and tested against the original version to 
assess the inter-rater reliability of each instrument. 

B. RESEARCH QUESTIONS 

The goal of this research was to determine if the perceived domain- 
specific gaps in DoD HFACS with respect to the surface Navy could be filled by 
creating a maritime specific version, HFACS-M. This study employed both 
HFACS taxonomies in conjunction with a case study to answer three questions. 

1. Research Question #1 

The first question addressed by this study is: Do SWOs and non-SWOs 
show the same consistency when applying DoD HFACS? HFACS was originally 
developed for application in naval aviation mishaps and has been amended and 
updated into its current version, DoD HFACS. The results of this study show a 
slightly higher Fleiss’ Kappa for non-SWOs at every level (nano-code, sub-code, 
and category) using DoD HFACS, than for SWOs. Recall that Fleiss’ Kappa is 
used to determine inter-rater reliability between a given number of raters. Fleiss’ 
Kappa indicates agreement between raters over that which could be reached by 
chance (Fleiss, 1971). Fleiss’ Kappa suffers from the fact that it does not have an 
agreed upon measure of significance, primarily because the number of subjects 
and categories directly impact the value (Gwet, 2010). Thus, it is not possible to 
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assign a particular meaning to a score (good, fair, bad, etc.). However, it can be 
stated that the non-SWOs’ higher scores when using DoD HFACS lead to the 
conclusion that a non-domain specific taxonomy yields a higher inter-rater 
reliability when employed with subjects not intimately familiar with the domain in 
question (maritime in this case). Again, based on the sample size and the fact 
that Fleiss’ Kappa was used, it is not possible to say that the difference was 
statistically significant. However, it is clear that, in this particular study, non- 
SWOs were more consistent when using DoD HFACS. This conclusion supports 
the findings of Wang et al. (2001), in which research showed rater agreement 
below 40% at the categorical level. Like the 2001 study, this study found that the 

group of raters applying DoD HFACS had a Kappa of just .391, or 39 %. It is 

interesting to note that when this group was divided into SWOs and non-SWOs, 
the non-SWOs had an inter-reliability of .436, some 10% higher than SWOs and 
4% higher than the group as a whole. Again, this result suggests that, when 
faced with a situation outside their scope of expertise (domain), subjects have a 
higher inter-rater reliability using a generic taxonomy. 

2. Research Question #2 

The second question addressed by this study is: What errors, overlaps, or 
gaps, if any, currently exist in DoD HFACS? Finding 7 from the NTSB accident 

report deals with the fatigue experienced by the pilot of one of the vessels 

involved. Nano-code PC307, Fatigue (sleep deprivation) was a commonly 
selected response, but PC308, Circadian rhythm de-synchronization, was also 
chosen by some subjects. PC308 seems to be a redundant code since a de- 
synchronization of an individual’s circadian rhythm causes fatigue. Thus, are 
these overlaps within the taxonomy? The truth about overlaps in HFACS seems 
to be: It depends. It depends on the person doing the investigation and the 
person assigning the codes based on the finding of the investigation. The 
wording of the investigation can have considerable impact on which selection the 
rater makes. Likewise, the training of the rater, along with his or her background 
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and expertise, all play pivotal roles in how the rater perceives the situation 
described in the investigation, and ultimately, which codes he or she will select. 

Where gaps are concerned, DoD HFACS does seem to lack domain 
specificity, as asserted by Griggs (2012) and Salmon et al. (2012). Corrective 
and preventive maintenance issues are extremely important in all branches of 
service, yet are not a part of DoD HFACS. It should be noted, however, that it 
would be impossible to create appropriate nano-codes for every minor error. To 
this end, domain specificity should not focus simply on what is missing, but also 
what makes one domain different from the next (i.e., the difference between 
submarines and aircraft), as both can lead to the discovery of gaps. 

Taking a broader look at DoD HFACS reveals an error classification 
taxonomy focused primarily on the event itself and not necessarily on latent 
errors. This emphasis on proximal errors rather than distal ones tends to 
eliminate potential latent errors from being identified. Manufacturing processes 
that produce hardware and software are less than perfect. Be it a mistake in a 
small string of code or a poorly welded seam, these errors can lie dormant for a 
large portion of the lifecycle of a ship, aircraft, or submarine until eventually the 
exact series of actions occur to cause them to be revealed in a catastrophic 
manner. Administrative processes that produce publications, instructions, and 
checklists are also prone to error. The incorrect wording of an emergent action in 
an instruction or the incorrect ordering of controlling actions for a casualty 
situation has the potential to cause more damage than they prevent. Issues such 
as these may be hard to identify during the course of an investigation and 
impossible to quantify without being properly addressed in the error classification 
instrument. 

3. Research Question #3 

The third question addressed by this study is: Does a tailored version of 
HFACS result in increased inter-rater reliability when classifying mishaps within 
the surface Navy? Why or why not? HFACS-M, the tailored maritime domain- 
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specific version of DoD HFACS showed higher overall Fleiss’ Kappa than DoD 
HFACS at every level of analysis (see Table 14). It is, however, not a staggering 
difference. FIFACS-M had a higher inter-rater reliability at the categorical level by 
5.7%, 7.4% at the sub-code level, and just 2.8% at the nano-code level. SWOs 
had a higher inter-rater reliability than non-SWOs when using HFACS-M at every 
level. In this study, SWOs had a higher inter-rater reliability than non-SWOs by 
10.9% at the categorical level, 7.4% at the sub-code level, and 6.5% at the nano¬ 
code level when using HFACS-M. Fleiss’ Kappa calculated for SWOs using 
HFACS-M were also higher than non-SWOs using DoD HFACS (by 4.5%, 6.9%, 
and .04% at the categorical, sub-code, and nano-code levels, respectively), 
which leads to the conclusion that subject matter experts (SWOs) have a slightly 
higher degree of agreement when using a domain specific instrument that 
employs terminology with which they are familiar. Based on the small sample 
size and untrained raters, however, further testing should be considered. 

The conclusion that domain specific error taxonomies produce higher 
inter-rater reliability when employed by subject matter experts appears to support 
what Salmon et al. (2011) and Griggs (2012) assert, “the taxonomy needs to be 
relevant to the maritime community” (Griggs, 2012, p. 85). In this study, HFACS- 
M, a domain-specific instrument, resulted in a slightly greater overall inter-rater 
reliability than the more generic DoD HFACS. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 

This research provides support for what Griggs (2012) asserted; that 
domain-specific error classification taxonomies, when employed by experts in 
that domain, may have greater accuracy than a generic or non-specific version. 
Greater specificity in error classification leads to more accurate hazard 
identification, which reduces mishaps in both quantity and severity. This finding is 
important for the Navy and DoD as a whole as fiscal constraints set in and yard 
periods and dry dock availabilities become fewer and farther between. 

It should be noted that the study was conducted with specific time 
constraints. The time to research and develop the apparatus and method 
spanned a six-month period. Although the subjects were experienced military 
officers, none had experience with HFACS outside of the brief training received 
immediately prior to reading the case study provided with this research. Despite 
these facts, it was still demonstrated that SWOs using HFACS-M displayed a 
slightly higher inter-rater reliability than non-SWOs. Fleiss’ Kappa calculated for 
HFACS-M was also slightly higher than that of DoD HFACS. 

B. RECOMMENDATIONS 

Based on the results of this study, future research should address the 
addition of nano-codes to address the previously discussed latent errors to 
ensure a much more robust taxonomy. HFACS-M added several nano-codes 
having to do with maintenance processes. Currently, a large gap exists in DoD 
HFACS, but both versions would benefit from nano-codes designed to account 
for latent distal errors accurately. The development of such codes would require 
extensive study but would add significantly to the body of knowledge surrounding 
human error and its classification and quantification within the DoD. 
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The findings of this study support the need for domain-specific human 
error taxonomies. However, the field of human error would benefit from more 
extensive research. A study using trained raters and a Naval Safety Center 
(NAVSAFCEN) mishap investigation from the surface Navy could help to validate 
the HFACS-M taxonomy. 

Finally, other domain-specific versions of HFACS should be developed 
and studied. To assume all branches of service and the communities therein 
have identical mishap potentials is to presume too much. The surface warfare 
community is far different from the aviation community, for example. While they 
can be generalized to a degree, at some point, the specific issues must be 
identified in the investigation process. These domain-specific issues can then be 
addressed so that the number of latent errors is reduced and the likelihood that 
an unfortunate chain of events will lead to a mishap is diminished. 
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APPENDIX A. HFACS TRAINING 



Human Error Classification 
Training 

LT Jason Bilbro 
Human Systems Integration 
Naval Postgraduate School 



• Mishaps cost the Department of Defense 
billions each year 

• Human error is involved in nearly every major 
mishap 

• Mishap prevention first requires identification 
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• Hazards find pathways through defenses and 
become losses 

• Humans are not perfect and neither are our 
defenses or prevention measures 



• Shappell and Weigmann created the HFACS 
taxonomy to turn theory into practice 

• Four categories: Organizational Influences, 
Supervision, Precondition, Acts 

• Each category is broken down into sub-codes 
with each sub-code into nano-codes 
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Organizational Influences 
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Preconditions 
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Example 


• The pilot of vessel A, the First Mate, had only 
six hours of sleep in the previous 48 hours due 
to watch rotation 

• The parent company of vessel A encouraged a 
schedule that maximized profits and did not 
allow for proper crew rest 

• The ship's master of vessel A routinely went 
without proper rest and did not encourage any 
different behavior among his crew 

• The pilot of vessel B did not adjust course in a 
timely manner to avoid the collision. 



• Carefully consider which of the four levels 
applies. 

- The applicable sub-codes within each level will 
help guide the nano-code selection 

• Many nano-codes are similar. Pay attention to the 
context of the findings to select the best choice. 
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• Below are the nano-codes associated with Organizational Influences. 
Take the time now and select the one which best describes the 
following: 

— The parent company of vessel A encouraged a schedule that maximized profits 
and did not allow for proper crew rest 


ORGANIZATIONAL INFLUENCE 

Nano Code 

Description 

Resource/Acquisition Management 

OK 001 

Air traffic control resources are deficient 

OR 00? 

Airfield resources are deficient 

OK 003 

Operational supporLfacililies/equipmenL are deficient 

OR 004 

Purshasing or providing poorly designed or unsuitable equipment 

OR 005 

Failure to remove inadequate/worn-out equipment in a timely manner 

OR 006 

Personnel recruiting and selection policies are inadequate 

OR 007 

Failure to provide adequeate manning/staffing resources 

OR 008 

railure to provi de adequeate operational informational resources 

OR 009 

Failure tn provi de adequate funding 

Organizational Climate 

OC 001 

Organizational culture {attitude/actions) allows for unsafe mission demand/pressure 

OC. 00? 

inappropriate perception nf promotion nr evaluation prnredures lead tn an unsafe art 

OC 003 

Organizational over-confidence or under-confidence in equipment 

Of. 004 

impending unit deartivatinn nr mission/equipnent change leads to unsafe situatinn 

OC 005 

Organizational structure is unclear or inadequate 

Organizational Process 

OP 001 

Pace of ops icmpo/workl oad creates unsafe si luation 

OP 002 

Organizational program/policy risks not adequately assessed, leading to an unsafe situation 

OP 003 

Provided inadequate procedural guidance or publications 

OP 004 

Organizational {formal} training is inadequate or unavailable 

OP 005 

Flawed doctiine/philosophy leads to unnecessary risks 

OP 006 

i nadequeate program management leads to unsafe situation 


rc fr T Organizational Influences (cont.) 


• The best selection in this case is: 

- OC 001 Organizational culture (attitude/actions) 
allows for unsafe mission demand/pressure 

• Alternative selections for this finding: 

OP 001 Pace of ops-tempo/workload creates unsafe 
situation is a tempting selection, but the issue at 
hand really speaks to an unsafe organizational 
climate that favors profits over safety. 
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• Below are the nano-codes associated with Supervision. Take the 
time now and select the one which best describes the following 

- The ship's master of vessel A routinely went without proper rest and did not 
encourage any different behavior among his crew 


SUPERVISION 

Nano Code 

Description 

inadequate Supervision 

SI 001 

Command oversight inadequate 

SI 002 

railed to ensure proper role-mocelirg 

SI 003 

Failed to provide proper training 

SI 004 

Failed to provide appropriate policy/guidance 

SI OOS 

Personality conflict with supervisor 

SI 006 

1 arit of supervisory responses to critical information 

Failure to Correct Known Problem 

SF001 

Failed to idcntify/correct risky behavior 

SF 002 

Failed to correct unsafe practices 

Planned inappropriate Operations 

SP001 

Directed mission beyond personnel/equipment capabilities 

SP 002 

Personnel mismatch 

SP 003 

Selected individual with lack of current experience 

SP 004 

Selected individual with limited overall experience 

SP 005 

Selected individual with lark of proficiency 

SP 006 

Performed inadequate risk assessment 

SP 00/ 

Authorized unnecessary hazard 

Supervisory Violations 

SV 001 

railure to enforoe existing rules 

SVOO? 

Allowing unwritten policies to become standard 

sv oca 

Directed individual to violate existing regulations 

5V 004 

Authorized unqualified individuals tor mission 


Supervision (cont.) 


• The best selection in this case is: 

- SI 002 Failed to ensure proper role-modeling 

• Alternative selections for this finding: 

— SF 001 Failed to identify/correct risky behavior and 
SF 002 Failed to correct unsafe practices are also 
reasonable selections, but the finding lends 
specifically to an issue with the way the ship's 
master allowed himself to be viewed and 
emulated. 
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Preconditions 


• Below are the nano-codes associated with Preconditions. Take the 
time now and select the one which best describes the following: 

— The pilot of vessel A, the First Mate, had only six hours of sleep in the 
_previous 48 hours due to the watch rotation 


Nano-Code | Description 

Physical Environment 

PE 101 

Id ng/fog on window restricts vision 

PE 102 

Weather conditions restricts vision 

PE 103 

VI brad ons affe ct v 1 slon or bal ance 

PE 104 

Ovst/smoke in workspace oostnjots vision 

PF ins 

Windhl.i-.t in win k-.p.m- obstructs vision 

PE 106 

Cold stress 

PE 107 

Heat st'ess 

PE ICQ 

Extiumu lora.'V limit jn individujl's movement 

pf irw 


PF 110 

NoiM* 

PF 11 1 

Hrnwnnut ivirl dnrmf/whifriKJt (i*x. -mow ilortnl 

technological Environment 

SF 001 

Sr.it and rrvli.iinl systems problems 

sf nn; 

In-.trumrnt.itinn and warningsysti-m issui-s 

SPOOl 

Visibility restrirtinns [notweathcr 

SF0O1 

Controls and switd-ies are inadequate 

SFOOS 

Automated system creates an unsafe situation 

sr 006 

Workspace Incompatible with operadon 

SF 007 

Personal equipment interference 

SF 008 

Communi cadon eq ui pment i nadeouate 

Self-Imposed Stress 

PP 201 

Physics' fitness level (Inapproprlateformlsslondemands 

PP 202 

Al coho' 

PP 203 

Drujts/cver-the-co unter medi cation/suppi e ments (not presai be d i 

PP 704 

Nulriliart/dirl 

PP 20S 

Inadec jate rest ise]fHmgosedi 

PP 206 

Operating with known disquial ify■ ng med ical conditi on 


Coord i nat ion/Commun i cati on/PI an. nin g Faa ors 

PP101 

Falure of crew/team leadership 

PP1IJ7 

F.vlurr>locrosschc*di/hark up 

PP 103 

Inadequate task del egad on 

PP104 

Bank/position md'rrvdation 

PT105 

Lack of assertiveness 

PP10G 

Critical information not commmuni cated 

PP107 

Standa-d/proper terminology not used 

PP1IX 

Falure lo ensure mmmunicalr-dintentions/actions wore understood .ind folio 

PPlOh 

Mssion planning inadeouate 

PP110 

Mssion briefing inadequate 

PP111 

Falure to re-assess risk and adjust tochana ngcircumstances 

PP 112 

Inf or .mad' on is misinterpreted or di sregarded 

Awareness (Cognitivo) Factors 

PC 101 

Not paying attention 

PC1U7 

Fi*ation {"channelized attention") 

pc ice 

Task over-satu'ation (e.g.. too much information to process) 

PC 101 

Confusion 

PC 105 

Negative transfer ie.g.. using old procedures for new system) 

PC10G 

EX str action 

pcio/ 

Geographically lost (confusion about location) 

PC1UB 

Interference/i nterrupd on during task 

Physical/Menta' Limitations 

PC401 

Learning rate limitations 

PC402 

Memory limitations 

PC403 

Gody size/movement limitations 

PC 404 

Coordination dnfionncy 

PC 405 

Technical or procedural knowledge not retained after training 


‘List of Preconditions continued on next page* 
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Preconditions 



• Below are the nano-codes associated with Preconditions. Take the 
time now and select the one that best describes the following: 

— The pilot of vessel A, the First Mate, had only six hours of sleep in the 
previous 48 hours due to watch rotation 


Pi'ru'plu.il F.idnr-. 

er.ioi 

Motion illu-.inn 

PC 502 

l jrm ne 4 lusion/baiance 

PC 501 


PC 504 

Mspercepfcon of chang.ng environment 

PC 505 

Mhinterpreted/mlsread Instrument (eg. misjudge altitude/distance/speed) 

PC 506 

innacurate expectati on (e.g.. see.ng/heanng what >s expected instead of what >s actually there/heai 

PC 50/ 

Misinterpretation of auditory cues 

PC 508 

Spafcal dl 

orientation - not recognized 

PC 500 

Spabal dl 

orientation - recognized 

PC 510 


orientation - Incapadtatlng 

PC 511 

Time distortion 



PC 201 

Pre-existing personality disorder'professionally diagnosed) 





pc7in 

Pri‘ ■•xj-.liriKp-.viliOSOCi.il problem (prr>li".-.irin.illv rli.iKnrr.nd) 

pcjivj 

Frnotian.ri -.l.iti- 

PC 205 

Personality -.t> r 

PC 206 

Overconfidence 

PC 707 

Pri-.-jriR (!•->:., pu-.hiriK self ur equipment Irin li.irdj 

PC 708 


PC 209 

Not enough motivation 

PC 210 

Misp:aced motivation 

PC 211 

More aggressive than necessary 

PC 212 

Excessive motivation to suosed [e.g„ "do or die") 

PC 211 

"Get-home-it-is"/"get-there-it-is" 

PC 214 

Inappropriate response due to expectation 

PC 215 

Motivational exhaustion ("burnout") 


Adverse I’sysiolo&cal btress 

PC 301 

effects of G forces (e.g.. G-LOC) 

pc: sv 

fffecls of prescribed drugs 

PC3CB 

Operational mjury/illness 

PC 304 

Sudden inrapar.itatinri/uneonxrinuxnexx(not duP to 6) 

PC 305 

Pre exisliogphvsiC.il illness/miurv 

PC 306 

Physical overexertion 

PC 307 

Fatigue (sleep deprivation) 

PC 308 

Circ.idi.io rhvthoide synch nom/.ilino (e.g.. iel l.v.or sliifl work) 

PC 300 

Motion sickness 

PC 310 

Trapped gas disorders 

PC 3)1 

rvolved gas disorders (e.g., decompression siekoess/bends) 

PC 312 

Reduced oxygen |hypoxia) 

PC 313 

llyperventilation{rapid breathing) 

PC 314 

ln.tdequ.it e adaptation to darkness 

PC 315 

Dehydration 

PC 31G 

Physical task over-saturation 
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• The best selection in this case is: 

- PC 307 Fatigue (sleep deprivation) 

• Alternative selections for this finding: 

- PP 205 Inadequate rest (self-imposed) seems like a 
good choice, but it was the watch rotation that 
took its toll, not a direct decision by the First Mate 

• This finding deals directly with adverse physiological 
stress, in this case fatigue, experienced by the First Mate. 



• Below are the nano-codes associated with Acts. Take the time now 
and select the one that best describes the following: 

— The pilot of vessel B did not adjust course in a timely manner to avoid the 
collision 



Nano Code 

Description 

Skill-Based Errors 

AF 1G1 

Unintended operation ot equipment 

AE 102 

Checklist not followed correctly 

AF 103 

Procedure not followed correctly 

Afc 104 

Over Controlled/undcr controlled aircraft/vehide 

AE 105 

Breakdown in visual scan 

AFICXi 

Inadequate Anti-G straining maneuver 

Judgement and Derision-Making Frrors 

AF201 

inadequate real-time risk assessment (e.g., failure of time-critical ORM) 

Afc202 

Failure to prioritise tasks adequately 

AE2Q3 

Rushed a necessary action 

AF704 

Delayed a necessary action 

AF7CR 

Ignored a raution/warriing 

AF 206 

Wrong choice of action during an operation (e.g., response to an emergency) 

Perception Errors 

AE301 

Incorrect response to a misperception (e.g., visual illusion or spatial disorientation) 

Violations 

AV001 

Work-around violation (e.g., breaking the rules is prceiveri as the best solution) 

Avon? 

Wiriespread/routine violation (e.g., habitual deviation from the rules that is tolerated by managemer 

AV 003 

Extreme violation (e.g., a violation not condoned by management 
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• The best selection in this case is: 

- AE 204 Delayed a necessary action 

• This finding deals directly with judgment and decision¬ 
making. In this case, the failure of the pilot to act in a 
timely manner directly caused the mishap 
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APPENDIX B. HFACS-M TRAINING 



Human Error Classification 
Training 

LT Jason Bilbro 
Human Systems Integration 
Naval Postgraduate School 



• Mishaps cost the Department of Defense 
billions each year 

• Human error is involved in nearly every major 
mishap 

• Mishap prevention first requires identification 
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• Hazards find pathways through defenses and 
become losses 


• Humans are not perfect and neither are our 
defenses or prevention measures 


DOD HFACS 


• Shappell and Weigmann created the HFACS 
taxonomy to turn theory into practice 

• Each category is broken down into sub-codes 
with each sub-code into nano-codes 

• HFACS-M looks at Organizational Influences, 
Command, Preconditions, and Acts 
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Organizational Influences 
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Example 


• The pilot of vessel A, the First Mate, had only 
six hours of sleep in the previous 48 hours due 
to watch rotation 

• The parent company of vessel A encouraged a 
schedule that maximized profits and did not 
allow for proper crew rest 

• The ship's master of vessel A routinely went 
without proper rest and did not encourage any 
different behavior among his crew 

• The pilot of vessel B did not adjust course in a 
timely manner to avoid the collision. 



• Carefully consider which of the four levels 
applies. 

-The applicable sub-codes within each level will 
help guide the nano-code selection 

• Many nano-codes are similar. Pay attention to the 
context of the findings to select the best choice. 
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• Below are the nano-codes associated with Organizational Influences. 
Take the time now and select the one which best describes the 
following: 

- The parent company of vessel A encouraged a schedule that maximized profits 
and did not allow for proper crew rest 


Nano-Code 

Description 

Resource/Acquisitipn Management 

Oft 001 

Port facilities are deficient 

Oft 002 

Channel markers/Iighting are deficient 

UR 008 

Opvr.vtion.il support f.iahtivs/pquipment.irp deficient 

OROoa 

Purshasingorproviding poorly designed or unsuitable equipment 

Oft DCS 

f.iil ure to procure now svslems/upgr.ides mat imvl v manner 

UR 00b 

failure to remove madequate/wom-out equipment in a timely manner 

Oft 007 

Personnel recruitingand selection oolides are inadeauate 

ur on 

failure toprovidv adequeatv m.wnmg/staffing resources 

Oft OOD 

Failure toprovide adequeate operational informational resources 

OR 010 

Fail urn to providv adequate funding 

ur on 

load equ at v quali tv assurance within m.v nt anance activity 1 v.ids to uns.ifv situ.it ion 

Organizational Climate 

OCOOl 

Uhgani/.ition.il culture (attitude/actions) .illows for unsafe missiondvm.ind/prvssurv 

IX WU 

Inappropn ate perception of promotion or evaluation procedures lead to an unsafe act 

OCOCQ 

Organizational over-confidence or under-confidence in equipment 

oc mu 

Impending umt dvactiv.ition or mission/equipment change lv.idstouns.ifv situation 

OC005 

Organizational structure is unclear orinadequate 

OrMni7.ntion.-vl Preau 

OT 001 

Pace of ops-tempo/workload creates unsafe situation 

OP 002 

Organizational program/policy risks not adequately assessed, leading to an unsafe situation 

UP OGS 

Provided inadequate iwocedur.il guidance or publications 

orooi 

Organizational (formal)training is inadequate orunavailable 

OP 005 

Flawed doctrine/philosophy leadstounnecessary risks 

OP 006 

Inadequeate program management leadsto unsafe situation 

OP 007 

Organizational process provides inadequate, untimely guidance 



• The best selection in this case is: 

- OC 001 Organizational culture (attitude/actions) 
allows for unsafe mission demand/pressure 

• Alternative selections for this finding: 

- OP 001 Pace of ops-tempo/workload creates 
unsafe situation is a tempting selection, but the 
issue at hand really speaks to an unsafe 
organizational climate that favors profits over 
safety. 
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Command 


• Below are the nano-codes associated with Command. Take the 
time now and select the one which best describes the following 

- The ship's master of vessel A routinely went without proper rest and did not 
encourage any different behavior among his crew 


Nano Code 

Description 

Inadequate Supervision 

a 001 

Inadequate oversight 

a aop 

F.vled to ensure proper role modeling. 

a oos 

Failed toprovide appropriate pollcy/gucdance 

ciooa 

Personalityconflictwfth supervisor 

aoco 

Lack of supervisoryresponsesto critical information 

a oub 

Failed to communicate intent<e.g., standingorders/mtfit orders! 

Failure toCorrect Known Problem 

cfooi 

Failed to identify/correct risky behavior 

CUD? 

Failed lo correct unsafe practices 

Manmng/Personnel/lrammglssues 

CM 001 

Directedmiss/on beyond oersonnel/eou pment capabi (ties 

CM (I>7 

Personnel mismatch 

CM 003 

Selected individual with lack of current experience 

CM 004 

Selected individual with limited overall experience 

CM 005 

Selected individual with lack of proficiency 

CM 006 

Directed mission without sufficient manning 

CM 007 

Command (formal) training inadequate 

CM 008 

Authorized unqualified individualsfor mission 


Planned Innappropnate operations 

CP 001 

Authorized unnecessary hazard 

CP 007 

Performed inadequate risk assessment (ORM) 

Supervisory Violations 

CVOOl 

Failure to enforce existing rules 

CV 002 

Allowing unwritten pol ides to become standard 

CV 003 

Directed individual to violate existing regulations 
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Command (cont.) 



• The best selection in this case is: 

- Cl 002 Failed to ensure proper role-modeling 

• Alternative selections for this finding: 

— CF 001 Failed to identify/correct risky behavior and 
CF 002 Failed to correct unsafe practices are also 
reasonable selections, but the finding lends 
specifically to an issue with the way the ship's 
master allowed himself to be viewed and 
emulated. 


.Jseb|L 


Preconditions 


• Below are the nano-codes associated with Preconditions. Take the 
time now and select the one which best describes the following: 


- The pilot of vessel A, the First Mate, had only six hours of sleep in the 
previous 48 hours due to the watch rotation 


Nano-Coae 

Oeserlpuon 

Ptiysrjl Friwrotimr-nl 

PC 101 

ic-ng/fog on w meow renncu visibility 

Pf in? 

Wi-athr-rrondillpriun-itrirtcvisibilnv 

Pfc 1U3 

Vibrations/rollsaffett vision or balance 

PE 10<1 

Dust/smoke i n workspace obstni as vision 

pf ins 

V.' rirltil.r.t In woritupurr obvtmrtc vision 

Pfc 106 

L»!d stress 

PC 107 

Ik: at stress 

PfclOH 

Extremeforces limit an mdiwdual's movement 

PfcllW 

Backlighbng/backscatter interfere with performance 

PC 110 

Noise 

Pfclll 

Brownout |e,g.. sand storm I/whiteout fe.g,. snow storm) 

PC 112 

Ughwlndj/Heaw seas affect/impair movement 

irrhnrdogx.’tl Fnvironmnnt 

sr 001 

instrumentation and warning system Issues 

sf no? 

Visibility rrstriHions |not vwathrr it-liitt*d) 

V- oos 

UMitroKurid uwilrbi's .«!»• iri.irtr-nd;«r- 

sr 004 

Automated system creates an unsafe situation 

SF OOS 

Workup,si inrompnliblr with npiT.ition 

SF 006 

Personal equlpmentinterference 

SF 007 

r/immiinimtiori rijinpmrnt iriadrijiiatr 

SFOOR 

Fqulpmr-nt not ronfiitumd rntrrrtly 

SF 008 

Corrective maintenance notconauaed/not conducted correctly 

voin 

Prrventativi- m.nntrnnnri' not roririnrlrri/riol n»nrtnrlr-rt rnrrrrlly 

ielf-lmposed Stress 

PP 201 

Pfiysi cal fitness level (iriaperopnate lor mission demands 

ppzub 

Alcohol 

PP2IH 

l>ugs/over-the-countermedication/supolements(not orestribedl 

PP21* 

Nutntion/diet 

PP20S 

Inadequate rest (veil imposed) 

PP2U6 

iterating w*h known disqualifyingmedical condition 


fnrxrlin.ttirjn/Ciimniuni iuitirin/PI-inrtinyj F.idnf. 

PP101 

r allure ol watcnteam/crew leadership 

PP 102 

Failure to croc:. uhudc/bjck up 

ppiitj 

In.irli'qu.ib* t.i'.k dpIrtMli nn 

PP1R4 

R.mk/pir,ili(>n intinurl.ition 

PP ll>» 

■ ark nf asseitivi>n(st 

PP 100 

Critical information not commmunicated 

PP 10/ 

Standard/proper terminology not used 

PP 108 

P.iiluii' In I'tv.uri- mnvmmiisiti-ri inlrnlirirv./.iiliniv. iwrri* undm.lnod .irwi frillrjwerl 

PP 109 

Mission planning inadequate 

PP 110 

Mssion briefing inadequate 

PP 111 

Failure to re-assess risk and adjust to changing orcumstances 

PP 112 

Information Is misinterpreted or disregarded 

Awareness (Cognitive) Factors 

PC 101 

Notpayi ng attention 

PC 102 

fixation ("channelized attention") 

PC 103 

Task over-saturation (e.g.. too much Information to process) 

PC 104 

Confusion 

PC 105 

Negative transfer |e.g„ using old procedures for new system) 

prins 

Di-.tr.idion 

PC 107 

C>i-riy;i.iphri:.il 1 y In'.t (miniu'.ion .ibnut Irxsilitjn) 

PC 106 

Interference/lnterruptlondurlngtask 

Ptw&iul/Mcntjl limitations 

PC 401 

lr.irnin>: rati* lirnit.itioiv. 

PC 407 

Mnmnry lirnlt.ili»iv. 

PC 403 

Body sirr/mnvrmrnHim>t.ition'. 

PC 404 

Coordination deficiency’ 

PC 405 

lechnical or procedural knowledge not retained after training 


‘List of Preconditions continued on next page* 
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• Below are the nano-codes associated with Preconditions. Take the 
time now and select the one that best describes the following: 

— The pilot of vessel A, the First Mate, had only six hours of sleep in the 
previous 48 hours due to watch rotation 


_ Perceptual factors _ 


Adverse Psysiological stress 

prsoi 

Ufalirm illusion 


PC 301 

Effects of prescribed drugs 

pr so? 

Turning lllu'.ian/b.il.ino' 


PC 302 

Operational injury/illness 

PC 503 

Visual illusion 


PC 303 

Sudden incapacitation/unconso'ousness 

PC SIM 

i\.«sperreptinn r.f rtianRiry; environment 


PC 354 

Pro existing physical illness/injury 

pesos 

istsinterpreted/misread instrument (e.g., misfudge altitude/distance/speed) 


PC 306 

PhySif.il OvPiVxPttiOrl 

prsnti 

lnn.icur.ili' nxpi'ct.itirin (I'.jr., •.iM'itWiM'.iriryK wh.it i\ i-npi'drrl irr.li'.irl r»l wh.il i'. .iilu.illy thrrr/hiMrrl 

PC 306 


PC 'Xi l 

lit (interpretation of aurito^ajes^ 


PC 307 

Orcadian rhythm de-synchroni tabon (watch rotation or shift work) 

pesos 

Spatial disorientation • notrecogmied 


PC 308 

f/otion sickness 

PC 'j09 

spatial disorientaton - recognited 


PC 309 


PC 510 

Spatial dlsorientation-lncapaotating 




PC 511 

Time distortion 


PC 311 

Inadequate adaptation to darioieu 

PC 201 



PC 317 

[Vh^dnitifM^ 

PC 202 

PC 203 

IVp-exIstlnr asurhasodil nrnhlem Inrofesslnnallv diaimnsedl' 




PC 209 

Cmotionel itile 



PC 70S 

Pi'c.nn.ilily -.tyli* 



PC 207 

Pressing (ex. push-ngself or equipment too haro) 



PC 708 

Cnmiil.iu'rtcy (r.>\. .ib'.i*na* nl wmr^__ 



pr 709 

Nnl ifiouRh molivjlion 



PC 710 

M'.pl.ia-rlmotiv.it ion 



PC 211 

P.tore aggressive than necessary 



PC 212 

excessive motivation to succeed (e.g., "do ordie|2 



PC 713 

•■C.-I hnr„r it iVTltct Ih.-rn it is" 



PC 714 

Inappropriate response due to expectation 



PC 215 

(\totivationai exhaustion ["burnout") 





• The best selection in this case is: 

- PC 306 Fatigue (sleep deprivation) 

• Alternative selections for this finding: 

- PP 205 Inadequate rest (self-imposed) seems like a 
good choice, but it was the watch rotation that 
took its toll, not a direct decision by the First Mate 

• This finding deals directly with adverse physiological 
stress, in this case fatigue, experienced by the First Mate. 
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• Below are the nano-codes associated with Acts. Take the time now 
and select the one that best describes the following: 

- The pilot of vessel B did not adjust course in a timely manner to avoid the 
collision 



Nano Code 

Description 

Skill Bitted Errors 

AE 101 

Unintended operation of equipment 

AF 10? 

Checklist not follow pd/nnt followed mire idly 

AF 103 

Procedure not fnllowed/not followed correctly 

AEioa 

Over Controlled or under controlled vessel 

Afc 105 

Breakdown in visual scan 

Judgement and Decision Making Errors 

AF?01 

Inadequate real-time risk assessment (e.g.. failure of time-critical ORM) 

AF 707 

Failure to prioritire tasks adequately 

AE 203 

Rushed d necessar^oction 

AE 204 

Delayed a necessary action 

AE 205 

Ignored a caution/waming 

AF ?06 

Wrong choice of action during an operation (e.g., response to an emergency) 

Perception Emm 

AE 301 

Incorrect response to a misperception (e.g.. visual illusion or spatial disorientation) 

Violations 

AV 001 

Work-around violation (e.g., breaking the mips is preeived as the hpst solution) 

AVOO? 

Widpspread/rnutine violation (e.g.. habitual deviation from the rules that istolerated by management) 

AV 003 

Extreme violation (e.g.. a violation not condoned by rn anagernent^ 



• The best selection in this case is: 

- AE 204 Delayed a necessary action 

• This finding deals directly with judgment and decision¬ 
making. In this case, the failure of the pilot to act in a 
timely manner directly caused the mishap 
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APPENDIX C. HFACS (EXCEL) 


Perceptual Factor* 


2 


4 


6 





11 

PC 501 

Motion illusion 












PC 502 

Turning illu^on/balance 












PC 503 













PC 504 

V .pw caption ol changing environment 












PC 505 

M'slnterpreted/misread instrument (e.g., mlsfudgealtitude/distance/speed) 












PC 506 

innacuratc ocpectation (eg., sccing/hcarfng what is expected instead of what is actually thcre/hcard) 












PC 507 

V.'vlntarpretation of auditory cue* 












PC 503 

Spatial disorientation • not reccgrVzed 












PC 509 

SpatJJ disorimtatfon • recognized 












PC 510 

Spatial disorientation - inrapadtatlng 












PC 511 

Time distortion 












Psycho-Batamvirf Factor* 













Pre-Listing personality dsorder (professionally dagnosed) 












PC 202 

Prc existing psychological dsorder (professionally d'agnoscdl 













Pre-existing psyrhosodal problem (professionally dlag noted) 












PC 204 

Emotional state 












PC 205 

Personality style 












PC 205 

Overconfidence 












PC 207 

Pressing (€*., pushing self or equipment too hard) 













Complacency (e.g., absence of worry) 












PC 209 

Not enough motivation 













V.'-placed motivation 













More aggressive than necessary 












PC 212 

Excestive motivation to succeed (eg.. "do or del 













"Get-hcm e2t4»’/"get-th«e-ilH*' 












PC 214 

inappropriate response due to expectation 













Motivational exhaustion ("burnout") 












Adverse Psysiologiod Strew 












PC 301 

Effects of G forces (e.g.. G-LOC) 












PC 202 

Effect* of preserbed drugs 












PC 403 

operational infury/nmess 












PC 304 

Sudden incapadtation/unconsdousrtcss (not due to G) 












PC 305 

Pr*-**.*tingphysci2 iHnew/Injury 












PC 306 

Pnysical over exer tlon 












PC 307 

Fatigue (sleep deprivation) 












PC3C6 

Orcadian rhythm de-synchronlzation (e.g.. /et leg or rhxlt work) 












PC 309 













PC 310 

Trapped gat dtorder* 












PC 311 

Evolved gas dsorders (eg.. decompression sickness/bends) 












PC 312 

Reduced oxygen [hypoxia] 












PC 313 

Hypoventilation (rapid breuth'rg) 












PC 314 

Inadequate adaptation to darkness 












PC 315 

Dehydration 












PC 316 

Physical task over-saturation 












SUPERVISION 
























Inadequate Supervision 












SI 001 

Command ouersght irvidcqjuU! 












SI 002 

Foiled to ensure proper role-modeling 












SI 003 

Failed to provide proper training 












SI 004 

Foiled to provide appropriate polity/guldance 












SI 005 

Personality conflict with supervisor 












SI 006 

Lack of supervisory responses to critical information 












Failure to Correct Known Problem 












SF 001 

Failed to fdentffy/correct risky behavior 












SF 002 

Foiled to correct unsafe practices 













Planned inappropriate Operations 












SP001 

Oircctcd mission beyond personnd/cqyipment capabilities 












SP 002 

P« vcrrxd rrmmatch 












SP 003 

Sheeted inrfvidual with lad: of current experience 












SP004 

Mected Indviduai with Kmited overall 












SP005 

SHected Indviduai with lack of profk it*ricy 












5P006 

Performed inadequate risk assessment 












SP007 

Author (red unxecetwy hatwd 












Supervisory Violations 












SVOOl 

Failure to enforce existing rules 












v ■ ; 

Allowing unwritten policies to become standard 












SV003 

Directed IrvJvidual to violate existing regulations 












SV004 

Authorized unqualified Individuals for mission 












ORGANIZATIONAL INFLUENCE 












Nano Code | Description 












Resource/Acquisition Management 












OR 001 

Air traffic control resourcesaredcffclent 












OR 002 

Airfield reiourcex are defidont 












OR 003 

Operational support fedllties/equipm ent are deficient 












OR 004 

Purchasing or providing poorly designed or unsuitable equipment 












OR 005 

Failure to remove Inedequate/wotn-out eqripmenl In a timely manner 












OR 006 

Personnel recruiting and select!cn policies are Inadequate 












OR 007 

Failure to provide adequeate mamlng/ttafflrg rewurce* 












OR 008 

F allure to provide adequeate operational informational resources 












OR 009 

Failure to provide adequate funding 












Organizational Climate 












OCOOl 

Organizational culture (attftude/actfons) allows for unsafe mission demand/pressure 












OC 002 

inappropriate perception of promotlonor evaluation procedure* load to an unsafe act 












OC 003 

Organizational over-confidence or under-confidence in equipment 












OC 004 

impending unit deactivation or misslon/cquipment charge leads to unsafe situation 












OC 005 

Organizational structure is undear or inadequate 












Organizational Process 












OP OOl 

Paoe ot ops t.snpq/wcrkload creates unsafe situation 












OP 002 

Organizational progrem/pollcy risks not adequately assessed, leadng to an unsafe situation 












OP 003 

Provided inadequate procedural guidance or publications 












OP 004 

Orgorirationol (formal) training Is inodeqiateor urvrvixlable 












OP 005 

Flawed dcctrlne/phllosophy leads to unnecessary risks 












OP 006 

Inadequeate program maragement leads to unsafe situation 
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APPENDIX D. HFACS-M (EXCEL) 


PlTCI-ptlJili FiJClOIS 

: 

: 

3 

4 

5 

6 

7 

R 

9 

10 

11 

PC SOI 

Mot inn illu-.icm 












VCW/ 

lurmne illusion/balance 












pesos 

Vi-ji ill illii'inri 












KCtzO-l 

Misperception of charging environment 












pc see 

Mlsinterprecec^misread instrument (eg., mlsiudge altitude/dstance/sgeed^^ 












KCtriXu 

innacurate expectation (eg., seeing/hearing what is expected nstead of what is actually there/heard] 












PCS07 

Misinterpretation of audtorycues 












pesos 

Sfiiil-.il ti-xiniiil;<ioii rml niay.nimi 












PCS09 

spat al dsonertat on - recognized 












pesio 

Sfiiils.il ri-uiruiil.tton iniiipuntutiiy. 












PC ill 

lime distortion 












Piyrho Betmviarui Factor*. 












PC 201 

Preexisting personally disorder (professionally diagnosed) 












PC 202 

Pie-exislir$( psyetiological disorder (professionally dagriosed^^ 












PC 203 

Preexisting psychosocial problem (professionall£diagnose^i 












PC 204 

Emotional state 












PC 20b 

Personality stylr 












PC 206 

Overconfidence 












PC 207 

Mrecong(ftfi, pushing set# nr equipment tooh vd) 












PC 208 

Complacency (ext. absence of worrv| 












PC 209 

Not ennigh motivation 












PC 210 

Misp’aced motivation 












PC 211 

Mon* :KEir-.'ivi'lh.inrii«|v^ir^^ 












PC 212 

Excessive motivation to succeed |e.g., 'do or die") 












PC 213 

•<5H home il !•.*/-««■* Ihiti-il i%* 












PC 214 

inaporoonate response due to expectation 












PC 21S 

Meltrv.ilicMiill rxhini-.liciri [’burnout*] 












Adverse Psysiological stress 












PC 301 

Effects of prescnbedjfrugs 












PC 302 

Uperational iniury/illness 












PC 303 

Sudden mcapaotatior/unconscousness 












PC 304 

Pn-t-xi-Zirp phy-jed ilbii-v./injuiv 












PC 3C6 

Physical overexertion 












pc: 3t» 

Kilyrur ('Jit-p ikfinviitmri) 












PC 30/ 

circadian rhythm de-synchronization (watch rotation or shift work) 












PC 306 

Motion sickness 












PC 311 

Reduced oxygen (hypoxia) 












PC 312 

• lyperventilatiun (rapid breaWnnj^^ 












PC 313 

inadequate adaptation to darkness 












PC 314 

Dehydration 












pc: HlIi 

Phyir-al t.vk nvrr yitnc.it.on 












COW MAW D 












Nano-Code | Oescntfio^^^ 












Inadequate Cuperviaon 












51001 

inadequate oversight 












SI 002 

railed to ensure proper role-modebng 












SI 003 

Fiilhtlto itoyicIi- ivupir ltninirij^^ 












SI 004 

railed to provide apprepnate poiicy/guid8nce 












SI Offs 

PiTion;flty conflict with •iipitvKor 












51000 

Lack of supervisory responses to critical information 












SI 007 

Fulhtf to cixnrnuriitiili- Inti'iil (r.g,\l. Hiding cxiIrtVniKhi ijicIit-.) 












Failure to correct Known Problem 












srooi 

railed to identify/correet risky behavior 












SP002 

Failedto correct unsafe practices 












Manning/Personnel/Trainirg Issues 












SP 001 

fui'CilNfmivuon kM'vaniJ im- r<<inniiiiriitt 1 laprlitlitu-. 












SP002 

Personnel mismatch 












SI* OCXS 

itHrcti-diridividual willi l.afc ufumiiit ixpiTNmr 












SP001 

3 elected l ndivl da al with limited overall experience 












sp cxi> 

iclrttrdindivitkj.il with InrJt of prof irarncy 












SP006 

Directed mission without sufficient mannlrg 












SP 007 

Command (formal] train inn inadecjuate 












SPO08 

Authonzed unqualified mdvldualsfor mission 













Planned Inri appropriate Operations 












SP 101 

Authonzed unnecessary hazard 












SP 102 

Piftciririi-d ni.ichtni.Cr ri-Jr itv.iv.mtnl (DRM) 












Supervisory Violations 












SV001 

railureto enforce existing rules 












SVU02 

Allowing unwritten policies to become standard 












SVC03 

DirectedIndMdual to violate existing regulations 












Ofifi/iNI/AIIOMAI INI IIIFNCI 












Nano-Ccxle | Description 












R('.iuri:i'/Aci|iir.itinri Minii^iirii'rit 












OK 001 

Port fat .ties are deficient 












moo? 

fhiMinrl iri.vkt-c.yiighling iiri-ck'fici itiI 












UK 003 

uperational support facihties/equipment are deficient 












CK004 

Purchaarg orprovidirg poorly designed or unsuitable equ pment 












OK 010 

F.iiliirrtn pronirrnrw '.v.livny/uivr.irk-'. in u limrlyrnunriiT 












CKCC5 

railureto remove inadequate/wom-out equipment in a timely manner 












OK cob 

PiTiaririif icuuilirp. iind •uHrtliciil iicCtcif. sin*imjjfirjnutr 












OK 007 

Tailuretu provide adequeaternaiimng/slBKing resources 












OKCXJK 

hililtiIK*to prnvtrh-. (Iii|ijisi1 ■'()|xt.< icitiid ■iftMimitinii.il trsourti". 












CK 033 

Failureto provide adequatefundng 












OR 010 

Inadequatequahtv assurance within iri air.tan ante activity leads to unsale situation 












Organizational Climate 












OCCOl 

Organ izutiunal culluru(altiludq/attions| aCows loi unsalomission dLTiiand/|xeviur^^ 












ex:CXI? 

inappropriate perception nf promotion nr evaluation procedures lead to an unsafe act 












OC 003 

OrganIcatiuriul uvur confidence or undur confidence in equipment 












OCOOt 

impendnp. unit deactivation nr mico on/equipment change leads to unsafe equation 












oeocs 

rirgiaijrutiiTi.il *.truCturi*i% undr.ii cm uiiidiquiitr 












Organizational Process 












OP 001 

P ace of ops-tempo/workload creates unsafe situation 












UP 002 

urgamzaticnal prqgram/pcficy nsks not aoequatefy assessed leading to an unsafe situation 












OP 003 

Provided inadequate procedural guidance or publications 












OP 004 

Organ it lit iinill (Inirriiil) 1 r. * r - r*r i*. in;idrqu;ilr tx uri;w;iiLihlr 












0PCC6 

flawed doctrine/ philosophy leads to unnecessary risks 












OP 006 

Iri.idiqur.ilc-pngr.im iriiiriifrciriin! Itiid. tci uiriili'iduiition 












UP 00/ 

urgamzaticnal process pr<wrdesinadecqate.untimely guidance 
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APPENDIX E. THESIS DATA 


A. NANO CODE 


DOD HFACS 

Findings 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

49/15 

AV001 

PP103 

AE202 

PP106 

PC307 

AE104 

PC 307 

OP003 

OP006 

OP002 

PE 204 

2 

1310 

PP103 

PP101 

PC108 

PP106 

PC307 

AE204 

PC 307 

OPOOl 

OCOOl 

SI001 

PP110 

3 

0602 

AE103 

AE103 

AE206 

PP106 

PC307 

AE206 

PC 307 

SI004 

SI004 

SF001 

PE 204 

4 

7565 

AE103 

PP101 

AE202 

PP106 

PC307 

AE204 

PC504 

SV002 

SI003 

SF002 

PE 204 

5 

49A 

PP103 

OP004 

PC102 

PP106 

PP205 

PC504 

PC404 

OP002 

OP003 

SI004 

PE206 

6 

21B/49 

AE202 

AV001 

PC106 

PP106 

SI001 

AE204 

PP205 

OR007 

OP005 

OP006 

OR004 

7 

1810 

PP103 

PP101 

PC106 

PP106 

PC307 

AE103 

AE201 

OCOOl 

51004 

SI001 

PE 204 

8 

1110 

AE103 

OP004 

PC108 

AE204 

PC308 

AE104 

PC 308 

SI004 

SF001 

SF002 

PE 204 

9 

1110 

5V001 

AE103 

AV001 

AE206 

OPOOl 

AE206 

OPOOl 

OP002 

OP005 

OP002 

OR004 

10 

1110 

SV002 

OP004 

AE103 

PP108 

PC308 

AE206 

PC 307 

SI001 

OP002 

OP006 

PE 204 

11 

1110 

AE103 

PP103 

PC106 

PP106 

PC308 

AE206 

OCOOl 

OCOOl 

OCOOl 

OP005 

PE204 

12 

1110 

AV001 

AE202 

AE202 

AE204 

PC307 

PP111 

PP205 

SI004 

SF002 

SI001 

PE204 

13 

1110 

AV003 

PP101 

AE206 

PP106 

PC305 

AE104 

PC 307 

OP002 

OP003 

OP006 

PE 204 

14 

1110 

AE202 

PP102 

SF001 

PP106 

PP206 

AE104 

PC 307 

SF001 

OP002 

OP006 

PE 207 















HFACS-M 

Findings 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

1317 

AE103 

AE201 

AE201 

PP106 

PC306 

AE104 

PC 306 

SI004 

SF002 

SI001 

PE203 

2 

1810 

AE103 

PP112 

PC106 

PP106 

PC304 

AE204 

PC 306 

SP201 

OP006 

OP002 

PE203 

3 

0602 

AV002 

SI001 

PC106 

AE204 

PC304 

PC101 

PP205 

OPOOl 

OP002 

OP006 

OR003 

4 

7523 

AE201 

AE103 

PP108 

AE206 

PC307 

AE204 

PC 306 

PE204 

PE202 

SI001 

OR004 

5 

1310 

AE103 

PP101 

AE202 

PP106 

PC306 

AE206 

PC306 

OP003 

SI004 

SI006 

OR004 

6 

1120 

AV001 

PP102 

AE203 

PP106 

OPOOl 

AE206 

PC 505 

OPOOl 

OR007 

OP006 

PE206 

7 

19A 

AV001 

PP102 

PP101 

PP106 

PC307 

AE204 

PC306 

SF001 

OP006 

SV002 

PE201 

8 

1110 

AV001 

PC206 

PP103 

PP106 

PC304 

AE204 

PC 307 

PC307 

OP002 

OCOOl 

PE201 

9 

1110 

AE103 

AE102 

PC108 

PP106 

PC307 

PP105 

PC 307 

OPOOl 

OP002 

OP006 

PE203 

10 

1110 

AE103 

PP101 

PC108 

PP106 

PC306 

AE104 

PC510 

SI004 

SF002 

SI001 

PE 208 

11 

1110 

AV001 

PC405 

PC102 

PP106 

PC307 

AE204 

PC 306 

OP003 

OP005 

OCOOl 

OR004 

12 

1110 

AE103 

AE206 

AE206 

AE204 

OPOOl 

AE204 

PC 306 

OP003 

OCOOl 

OP007 

PE203 

13 

1110 

AE103 

PP101 

PC108 

PP106 

PC306 

AE204 

PC 306 

OP002 

OP005 

SI004 

OP006 

14 

1110 

PC306 

PP101 

PP108 

PP106 

PC306 

AE204 

PC 307 

OCOOl 

SP007 

SI001 

OR004 
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B. SUB CODE 


DOD HFACS 

Findings 

Number 

Desig/MOS/AFSC 

i 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

i 

49/15 

A-V 

P-CCPF 

A-JDME 

P-CCPF 

P-APS 

A-SB 

P-APS 

O-P 

O-P 

O-P 

P-TE 

2 

1310 

P-CCPF 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

O-C 

S-IS 

P-CCPF 

3 

0602 

A-S8 

A-SB 

A-JDME 

P-CCPF 

P-APS 

A-JDME 

P-APS 

S-IS 

S-IS 

S-FCKP 

P-TE 

4 

7565 

A-SB 

P-CCPF 

A-JDME 

P-CCPF 

P-APS 

A-JDME 

P-PF 

S-SV 

S-IS 

S-FCKP 

P-TE 

5 

49A 

P-CCPF 

O-P 

P-AF 

P-CCPF 

P-SIS 

P-PF 

P-PML 

O-P 

O-P 

S-IS 

P-TE 

6 

216/49 

A-JDME 

A-V 

P-AF 

P-CCPF 

S-IS 

A-JDME 

P-SIS 

O-RAM 

O-P 

O-P 

O-RAM 

7 

1810 

A-S8 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-SB 

A-JDME 

O-C 

S-IS 

S-IS 

P-TE 

8 

1110 

A-S8 

O-P 

P-AF 

A-JDME 

P-APS 

A-SB 

P-APS 

S-IS 

S-FCKP 

S-FCKP 

P-TE 

9 

1110 

S-SV 

A-SB 

A-V 

A-JDME 

O-P 

A-JDME 

O-P 

O-P 

O-P 

O-P 

O-RAM 

10 

1110 

S-SV 

O-P 

A-SB 

P-CCPF 

P-APS 

A-JDME 

P-APS 

S-IS 

O-P 

O-P 

P-TE 

11 

1110 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-JDME 

o-c 

o-c 

o-c 

O-P 

P-TE 

12 

1110 

A-V 

A-JDME 

A-JDME 

A-JDME 

P-APS 

P-CCPF 

P-SIS 

S-IS 

S-FCKP 

S-IS 

P-TE 

13 

1110 

A-V 

P-CCPF 

A-JDME 

P-CCPF 

P-APS 

A-SB 

P-APS 

O-P 

O-P 

O-P 

P-TE 

14 

1110 

A-JDME 

P-CCPF 

S-FCKP 

P-CCPF 

P-SIS 

A-SB 

P-APS 

S-FCKP 

O-P 

O-P 

P-TE 















HFACS-M 

Findings 

Number 

Desig/MOS/AFSC 

i 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

1317 

A-SB 

A-JDME 

A-JDME 

P-CCPF 

P-APS 

A-SB 

P-APS 

C-IS 

C-FCKP 

C-IS 

P-TE 

2 

1810 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

C-PIO 

O-P 

O-P 

P-TE 

3 

0602 

A-V 

C-IS 

P-AF 

A-JDME 

P-APS 

P-AF 

P-SIS 

O-P 

O-P 

O-P 

O-RAM 

4 

7523 

A-JDME 

A-SB 

P-CCPF 

A-JDME 

P-APS 

A-JDME 

P-APS 

P-TE 

P-TE 

C-IS 

O-RAM 

5 

1310 

A-SB 

P-CCPF 

A-JDME 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

C-IS 

C-IS 

O-RAM 

6 

1120 

A-V 

P-CCPF 

A-JDME 

P-CCPF 

O-P 

A-JDME 

P-PF 

O-P 

O-RAM 

O-P 

P-TE 

7 

19A 

A-V 

P-CCPF 

P-CCPF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

C-FCKP 

O-P 

c-sv 

P-TE 

8 

1110 

A-V 

P-P8F 

P-CCPF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

O-P 

o-c 

P-TE 

9 

1110 

A-SB 

A-SB 

P-AF 

P-CCPF 

P-APS 

P-CCPF 

P-APS 

O-P 

O-P 

O-P 

P-TE 

10 

1110 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-SB 

P-PF 

C-IS 

C-FCKP 

C-IS 

P-TE 

11 

1110 

A-V 

P-PML 

P-AF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

O-P 

o-c 

O-RAM 

12 

1110 

A-SB 

A-JDME 

A-JDME 

A-JDME 

O-P 

A-JDME 

P-APS 

O-P 

O-C 

O-P 

P-TE 

13 

1110 

A-SB 

P-CCPF 

P-AF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-P 

O-P 

C-IS 

O-P 

14 

1110 

P-APS 

P-CCPF 

P-CCPF 

P-CCPF 

P-APS 

A-JDME 

P-APS 

O-C 

C-MPT 

C-IS 

O-RAM 
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C. DOD HFACS SUB 


ASB | AIDMf | APT | AV | P Pf | P Tf | P SIS | PCCPf | P AF | P PMI | P Pf \ P PBf | P APS | SIS |SFCKP|SPIO| S SV | O RAM | PC 



n| loj ol s| ul n| 


■I 'I °l 


»l 'I °l ’I ■! S T" 


0.123 0.000 0.032 0.000 0.071 0.026 0.143 0.039 0.006 0 013 0 000 0.110 0.0/8 0.045 0.000 0.019 0.019 0.032 0.156 


Pbar = 
PcBar = 
Kappa = 


0.352 

0.098 

0.281 


ASB | A IDMf | A PF | A V | P PF | P T1 | P SIS | PCCPI | P Af | P PMI | P PF | P PBf | P APS | S IS | S ICKP| S PIO | S SV | O RAM | PC | QP~ 


DOD SWO SUB 


g 'l. 




77 


0.091 0.130 0.000 0 039 0 000 0.078 0.026 0.104 0.026 0.000 0 000 0 000 0.117 0.052 0.065 0.000 0.026 0.013 0.039 0.195 




Pbar = 
PcBar = 
Kappa= 


0.329 

0.106 

0.250 


DOD NON SWO SUB 

A SB | A ID Ml | 

A PI | 

AAV | 

P PE | 

Pit | 

P SIS | 

PCCPF | 

P AF | 

P PMI | 

P Pf | 

P PBf | 

P APS | 

sis Is fckpI 

S PIO | 

s sv | 

o ram| 

oc I 

OP 

PI 


1 

3 1 

0 

1 

0 

0 

0 

2 

0 

0 

0 

0 

0 

0 0 

0 

0 

0 

0 

0 


7 

2 

I 0 

0 

1 

0 

0 

0 

4 

0 

0 

0 

0 

0 

0 0 

0 

0 

0 

0 

1 


7 

3 

0 3 

0 

0 

0 

0 

0 

0 

4 

0 

0 

0 

0 

0 0 

0 

0 

0 

0 

0 

0.429 

7 

4 

0 0 

0 

0 

0 

0 

0 

7 

0 

0 

0 

0 

0 

0 0 

0 

0 

0 

0 

0 


7 

5 

0 0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

5 

1 0 

0 

0 

0 

0 

0 


7 

6 

7 4 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 0 

0 

0 

0 

0 

0 

0 333 

7 

7 

0 1 

0 

0 

0 

0 

1 

0 

0 

1 

1 

0 

3 

0 0 

0 

0 

0 

0 

0 

0.143 

7 

8 

0 0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 0 

0 

1 

1 

1 

3 

0 143 

7 

9 

0 0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3 0 

0 

0 

0 

1 

3 

0 286 

7 

10 

0 0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3 2 

0 

0 

0 

0 

2 

0738 

7 

11 

0 0 

0 

0 

0 

5 

0 

1 

0 

0 

0 

0 

0 

0 0 

0 

0 

1 

0 

0 

04/6 

7 

Total 

6| 9| 

°l 

A 

°l 

A 

A 

14| 

4 I 

A 

>1 

°l 

8 I 

8| 2| 

°l 

>1 

>1 

A 

9 

4 000 

77 

Pi 

0.078 0.117 

0.000 

0026 

0000 

0.065 

0.026 

0 182 

0.052 

0 013 

0026 

0 000 

0.104 

0 104 0 026 

0000 

0.013 

0 026 

0026 

0.11/ 




Pbar 

PeBar 

Kappa 


0.364 

0.099 

0.293 
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D. HFACS-M SUB 





Mmi - 
PcBjr = 
ICappj 


0.564 

OIOS 

0.289 
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E 


CATEGORICAL 


DOD HFACS 

F 

'ridings 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

49/15 

A 

P 

A 

P 

P 

A 

P 

0 

0 

0 

p 

2 

1310 

P 

P 

P 

P 

P 

A 

P 

0 

0 

0 

p 

3 

0602 

A 

A 

A 

P 

P 

A 

P 

0 

0 

0 

p 

4 

7565 

A 

P 

A 

P 

P 

A 

P 

S 

S 

s 

p 

5 

49A 

P 

0 

P 

P 

P 

P 

P 

0 

0 

s 

p 

6 

21B/49 

A 

A 

P 

P 

5 

A 

P 

0 

0 

0 

0 

7 

1810 

A 

P 

P 

P 

P 

A 

A 

0 

5 

s 

p 

8 

1110 

A 

0 

P 

A 

P 

A 

P 

S 

S 

s 

p 

9 

1110 

S 

A 

A 

A 

0 

A 

0 

0 

0 

0 

0 

10 

1110 

5 

0 

A 

P 

P 

A 

P 

S 

0 

0 

p 

11 

1110 

A 

P 

P 

P 

P 

A 

0 

0 

0 

0 

p 

12 

1110 

A 

A 

A 

A 

P 

P 

P 

S 

S 

s 

p 

13 

1110 

A 

P 

A 

P 

P 

A 

P 

0 

0 

0 

p 

14 

1110 

A 

P 

S 

P 

P 

A 

P 

S 

0 

0 

p 















HFACS-M 

F 

'ndings 

Number 

Desig/MOS/AFSC 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 

1317 

A 

A 

A 

P 

P 

A 

P 

C 

C 

c 

P 

2 

1810 

A 

A 

P 

P 

P 

A 

P 

C 

0 

0 

P 

3 

0602 

A 

C 

P 

A 

P 

P 

P 

0 

0 

0 

0 

4 

7523 

A 

A 

P 

A 

P 

A 

P 

P 

P 

c 

0 

5 

1310 

A 

P 

A 

P 

P 

A 

P 

0 

C 

c 

0 

6 

1120 

A 

P 

A 

P 

0 

A 

P 

0 

0 

0 

P 

7 

19A 

A 

P 

P 

P 

P 

A 

P 

C 

0 

c 

P 

8 

1110 

A 

P 

P 

P 

P 

A 

P 

0 

0 

0 

P 

9 

1110 

A 

A 

P 

P 

P 

P 

P 

0 

0 

0 

P 

10 

1110 

A 

A 

P 

P 

P 

A 

P 

C 

c 

c 

P 

11 

1110 

A 

P 

P 

P 

P 

A 

P 

0 

0 

0 

0 

12 

1110 

A 

A 

A 

A 

0 

A 

P 

0 

0 

0 

P 

13 

1110 

A 

A 

P 

P 

P 

A 

P 

0 

0 

c 

0 

14 

1110 

P 

P 

P 

P 

P 

A 

P 

0 

c 

c 

0 
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F 


DOD HFACS CATA 


DOD CAT 

0 

S 

P 

A 

Pi 

1 

0 

2 

2 

10 

0.516 

2 

3 

0 

7 

4 

0.330 

3 

0 

1 

6 

7 

0.396 

4 

0 

0 

11 

3 

0.637 

5 

1 

1 

12 

0 

0.725 

6 

0 

0 

2 

12 

0.736 

7 

2 

0 

11 

1 

0.615 

8 

9 

5 

0 

0 

0.505 

9 

10 

4 

0 

0 

0.560 

10 

9 

5 

0 

0 

0.505 

11 

2 

0 

12 

0 

0.736 

Total 

36 

18 

63 

37 

6.264 


Pi 

0.234 

0.117 

0.409 

0.240 

Pbar = 

0.569 




PeBar = 

0.293 




Kappa = 

0.391 





DD SWO C/ 

O 

S 

P 

A 

Pi 

1 

0 

2 

0 

5 

0.524 

2 

2 

0 

3 

2 

0.238 

3 

0 

1 

2 

4 

0.333 

4 

0 

0 

4 

3 

0.429 

5 

1 

0 

6 

0 

0.714 

6 

0 

0 

1 

6 

0.714 

7 

2 

0 

5 

0 

0.524 

8 

3 

4 

0 

0 

0.429 

9 

5 

2 

0 

0 

0.524 

10 

5 

2 

0 

0 

0.524 

11 

1 

0 

6 

0 

0.714 

Total 

19 

11 

27 

20 

5.667 


Pj 

0.247 

0.143 

0.351 

0.260 

Pbar = 

0.515 




PeBar = 

0.272 




Kappa = 

0.334 





NON-SWO 

O 

S 

P 

A 

Pi 

1 

0 

0 

2 

5 

0.524 

2 

1 

0 

4 

2 

0.333 

3 

0 

0 

4 

3 

0.429 

4 

0 

0 

7 

0 

1.000 

5 

0 

1 

6 

0 

0.714 

6 

0 

0 

1 

6 

0.714 

7 

0 

0 

6 

1 

0.714 

8 

6 

1 

0 

0 

0.714 

9 

5 

2 

0 

0 

0.524 

10 

4 

3 

0 

0 

0.429 

11 

1 

0 

6 

0 

0.714 

Total 

17 

7 

36 

17 

6.810 

Pj 

0.221 

0.091 

0.468 

0.221 



Pbar = 0.619 

PeBar = 0.324 

Kappa = 0.436 
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G 


HFACS-M CATA 


M CAT 

0 

C 

P 

A 

Pi 

1 

0 

0 

1 

13 

0.857 

2 

0 

1 

6 

7 

0.396 

3 

0 

0 

10 

4 

0.560 

4 

0 

0 

11 

3 

0.637 

5 

2 

0 

12 

0 

0.736 

6 

0 

0 

2 

12 

0.736 

7 

0 

0 

14 

0 

1.000 

8 

9 

4 

1 

0 

0.462 

9 

9 

4 

1 

0 

0.462 

10 

7 

7 

0 

0 

0.462 

11 

6 

0 

8 

0 

0.473 

Total 

33 

16 

66 

39 

6.780 

Pj 

0.214 

0.104 

0.429 


Pbar = 0.616 

PeBar = 0.305 

Kappa = 0.448 

M SWO CAT 

O 

C 

P 

A 

Pi 

1 

0 

0 

1 

6 

0.714 

2 

0 

0 

3 

4 

0.429 

3 

0 

0 

6 

1 

0.714 

4 

0 

0 

6 

1 

0.714 

5 

1 

0 

6 

0 

0.714 

6 

0 

0 

1 

6 

0.714 

7 

0 

0 

7 

0 

1.000 

8 

6 

1 

0 

0 

0.714 

9 

5 

2 

0 

0 

0.524 

10 

4 

3 

0 

0 

0.429 

11 

3 

0 

4 

0 

0.429 

Total 

19 

6 

34 

18 

7.095 

Pj 0.247 0.078 0.442 

Pbar= 0.645 

PeBar = 0.317 

Kappa = 0.481 


M NON-SWOCAT 

O 

C 

P 

A 

Pi 

1 

0 

0 

0 

7 

1.000 

2 

0 

1 

3 

3 

0.286 

3 

0 

0 

4 

3 

0.429 

4 

0 

0 

5 

2 

0.524 

5 

1 

0 

6 

0 

0.714 

6 

0 

0 

1 

6 

0.714 

7 

0 

0 

7 

0 

1.000 

8 

3 

3 

1 

0 

0.286 

9 

4 

2 

1 

0 

0.333 

10 

3 

4 

0 

0 

0.429 

11 

3 

0 

4 

0 

0.429 

Total 

14 

10 

32 

21 

6.143 

Pj 

0.182 

0.130 

0.416 

0.273 



Pbar = 0.558 

PeBar = 0.297 

Kappa = 0.372 
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H. 


OVERALL ANALYSIS 



DOD HFACS 

HFACS-M 

swo 

7 

7 

NON-SWO 

7 

7 



Overall 

SWO 

NON-SWO 

DOD HFACS (Nano-code) 

0.154 

0.114 

0.198 

HFACS-M (Nano-code) 

0.182 

0.202 

0.137 

DOD HFACS (Sub-code) 

0.281 

0.25 

0.293 

HFACS-M (Sub-code) 

0.355 

0.362 

0.289 

DOD HFACS (Category) 

0.391 

0.334 

0.436 

HFACS-M (Category) 

0.448 

0.481 

0.372 
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